CodeCell: Ai Prompt

In this build, we'll explore how to configure the CodeCell's ESP32-C3 to use Google's Gemini AI for Arduino prompt-based interactions. You'll learn how to send a prompt via the Serial Monitor and, in a second example, how the CodeCell can automatically trigger a joke based on proximity sensing. This project is ideal for anyone looking to add AI capabilities to their IoT projects.

What You'll Learn

How to set up the CodeCell for AI prompt-based interactions via the ESP32-C3.
How to send prompts to Gemini AI using the Arduino IDE and Serial Monitor.
How to use CodeCell’s proximity sensor to trigger a joke automatically.

About CodeCell and Google Gemini

In this example, we use Google’s Gemini model for generating content based on user input or sensor data. Through out this tutorial we will use and modify the code example made by 'techiesms' - Watch the full tutorial here.

With the ESP32-C3's WiFi capabilities, you can make HTTP requests to Google’s Gemini API, allowing real-time interaction with the AI. Whether you’re asking for text responses or generating creative outputs like jokes, this integration is straightforward to implement.

Project Overview

In the first example, you’ll send prompts directly via the Serial Monitor, and the CodeCell will send this input to Google Gemini AI for processing. The AI's response is printed back to the Serial Monitor, limited by 100 tokens. In the second example, the CodeCell’s proximity sensor will trigger a prompt to the AI, asking it to generate a joke when it detects an object. This setup can be used for fun interactive projects where the device responds to its environment using AI-based content.

How to Get the Gemini API Key

Before we integrate the Gemini AI into our ESP32-C3 setup, we first need to generate an API key and test it. Follow the steps below to create your API key and then you can also test it using a software like Postman.

Step 1: Generating the API Key for Google Gemini

Open your browser and search for Gemini API
Click on the first result that appears. This will take you to Google’s API documentation page.
Next, click on the button that says 'Get API Key in Google AI Studio', then click 'Create API Key'.
Once the API key is generated, copy it somewhere safe. You will need this for the next steps.

Step 2: Testing the API with Postman

Now that we have the API key, we can test it using the Postman application. Postman is a free tool that allows you to make HTTP requests and see the responses.

First, download and install Postman from the official website. You can find the download link here.
After installation, open Postman and create a new account if required.
Once you're logged in, click the + icon to create a new request.
Select POST as the request type, as we will be sending a prompt to the Gemini AI.
For the URL, modify this URL with your API Key, you generated in Step 1: https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent?key=YOUR_API_KEY

Step 3: Configuring Headers and Body in Postman

Once you’ve entered the URL, we need to set up the request headers and body.

Go to the Headers section in Postman and add a new header.
Set the header key to Content-Type and the value to application/json.
Now, click on the Body tab and select raw and set the format to JSON.
In the body, paste the following JSON code:


{
    "contents": [
        {
            "parts": [
                {
                    "text": "Who are you?"
                }
            ]
        }
    ],
    "generationConfig": {
        "maxOutputTokens": 100
    }
}

In this example, we are asking the AI a simple question: "Who are you?" and setting the maximum number of tokens to 100. Tokens control the length of the response generated by the AI. If you lower the token limit (e.g., 20 tokens), the response will be shorter. You can experiment with different values for maxOutputTokens to see how it affects the response length.

Step 4: Sending the Request

Once everything is set up, click the Send button.
Postman will send the request to the Gemini AI, and after a few seconds, you should see a response.
If everything was successful, you’ll see a status of 200 OK, indicating that the request was processed without errors.
The response body will contain the answer generated by the AI. In this case, you should see something like: "I am a large language model trained by Google."

How to prompt with CodeCell?

Once you've generated and verified that the API works, you can proceed with the next step: integrating this API into your CodeCell project.

Example 1: AI Prompt via Serial Monitor

Below is the example code to get you started. In this example, the AI will respond to text prompts you send via the Serial Monitor. Remember to replace the placeholders with your WiFi credentials and Gemini API token.


#include <Arduino.h>
#include <WiFi.h>
#include <HTTPClient.h>
#include <ArduinoJson.h>
#include <CodeCell.h>

CodeCell myCodeCell;

const char* ssid = "SSID"; //Enter your SSID
const char* password = "PASSWORD"; //Enter your password
const char* Gemini_Token = ""; //Enter your Gemini token
const char* Gemini_Max_Tokens = "100";
String res = "";

void setup() {
  Serial.begin(115200);

  WiFi.mode(WIFI_STA);
  WiFi.disconnect();

  while (!Serial);

  // Wait for WiFi connection
  WiFi.begin(ssid, password);
  Serial.print("Connecting to ");
  Serial.println(ssid);
  while (WiFi.status() != WL_CONNECTED) {
    delay(1000);
    Serial.print(".");
  }
  Serial.println("connected");
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());
}

void loop() {
  while (!Serial.available());

  while (Serial.available()) {
    char add = Serial.read();
    res += add;
    delay(1);
  }

  int len = res.length();
  res = res.substring(0, len - 1);
  res = "\"" + res + "\"";

  HTTPClient https;

  if (https.begin("https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent?key=" + String(Gemini_Token))) {
    https.addHeader("Content-Type", "application/json");
    String payload = "{\"contents\": [{\"parts\":[{\"text\":" + res + "}]}],\"generationConfig\": {\"maxOutputTokens\": " + String(Gemini_Max_Tokens) + "}}";

    int httpCode = https.POST(payload);

    if (httpCode == HTTP_CODE_OK) {
      String response = https.getString();
      DynamicJsonDocument doc(1024);
      deserializeJson(doc, response);
      String answer = doc["candidates"][0]["content"]["parts"][0]["text"];
      answer.trim();

      Serial.println(answer);
    } else {
      Serial.printf("[HTTPS] POST failed, error: %s\n", https.errorToString(httpCode).c_str());
    }
    https.end();
  } else {
    Serial.printf("[HTTPS] Unable to connect\n");
  }

  res = "";
}

Example 2: Trigger AI Prompt via Proximity Sensor

This example uses the CodeCell's proximity sensor to trigger a prompt when an object is detected nearby. The AI will respond with a joke.


#include <Arduino.h>
#include <WiFi.h>
#include <HTTPClient.h>
#include <ArduinoJson.h>
#include <CodeCell.h>

CodeCell myCodeCell;

const char* ssid = "SSID"; //Enter your SSID
const char* password = "PASSWORD"; //Enter your password
const char* Gemini_Token = ""; //Enter your Gemini token
const char* Gemini_Max_Tokens = "100";
String res = "";

void setup() {
  Serial.begin(115200);

  myCodeCell.Init(LIGHT);  // Initializes proximity sensing

  WiFi.mode(WIFI_STA);
  WiFi.disconnect();

  while (!Serial);

  WiFi.begin(ssid, password);
  Serial.print("Connecting to ");
  Serial.println(ssid);
  while (WiFi.status() != WL_CONNECTED) {
    delay(1000);
    Serial.print(".");
  }
  Serial.println("connected");
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());
}

void loop() {
  if (myCodeCell.Run()) {
    uint16_t proximity = myCodeCell.Light_ProximityRead();

    if (proximity > 100) {
      Serial.println("Here's a new joke...");
      myCodeCell.LED(0, 0xFF, 0);  // Set LED to Green when proximity is detected

      res = "\"Tell me a unique joke\"";

      HTTPClient https;

      if (https.begin("https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent?key=" + String(Gemini_Token))) {
        https.addHeader("Content-Type", "application/json");
        String payload = "{\"contents\": [{\"parts\":[{\"text\":" + res + "}]}],\"generationConfig\": {\"maxOutputTokens\": " + String(Gemini_Max_Tokens) + "}}";

        int httpCode = https.POST(payload);

        if (httpCode == HTTP_CODE_OK) {
          String response = https.getString();
          DynamicJsonDocument doc(1024);
          deserializeJson(doc, response);
          String answer = doc["candidates"][0]["content"]["parts"][0]["text"];
          answer.trim();

          Serial.println(answer);
        } else {
          Serial.printf("[HTTPS] POST failed, error: %s\n", https.errorToString(httpCode).c_str());
        }
        https.end();
      } else {
        Serial.printf("[HTTPS] Unable to connect\n");
      }

      res = "";
    }
  }
}

Tips for Customization

Different Prompts: Try customizing the prompts to ask the AI different questions or give commands for creative outputs.
Experiment with Other Sensors: You can trigger different AI responses based on input from other CodeCell sensors like motion or light sensing.

Conclusion

This project showcases how to integrate AI responses into your CodeCell projects using Google’s Gemini API. By leveraging the ESP32-C3’s WiFi capabilities, you can create interactive devices that react to user input or environmental factors, making your IoT builds smarter and more engaging.

Experiment with the code and customize the prompts to suit your projects!