IFD:Designing Networked Objects/César Felipe Daher

Twitter-Controlled LED Matrix

The concept

This project was created as an attempt to use Twitter API to send commands to a microcontroller. Its basic structure consists of a matrix of LEDs in which only one of them is lit up at a time. The Twitter API is used to collect information from tweets in order to choose which LED to light up.

At first, the idea was to collect and analyze information from tweets all around the world, so as to generate inputs. However, as this was my first time working with both microcontrollers and APIs, it was preferred to lower the complexity of this interaction to the level of generating inputs from individual tweets using the search feature.

Requirements

The following materials and services were used in order to develop this project:

• ESP8266 NodeMCU microcontroller;
• Elegoo UNO R3 Super Starter Kit (or similar, such as Arduino Uno);
  • 9 LEDs;
  • 9 resistors (220 Ohms);
  • Jumper wires;
  • Breadboard;
• Arduino software;
• Twitter API Library for Arduino;
• Twitter developer account;

The core of this project is the use of ESP8266 in order to connect to the internet and access the Twitter API. It is worth mentioning, though, that I was required to apply for a developer Twitter account in order to access the API.

References

In order to develop this project, most of my research was focused on tweet-to-microcontroller interactions on platforms such as Pinterest, Instructables and Hackaday. The two examples below are, respectively, Mood Lamp Based on Twitter Hashtags and Twitter-Controlled Pet Feeder. The former searches for the use of hashtags such as #angry in order to determine the color of the lamp, while the latter receives from a specific user tweets as direct commands to the pet feeder.

• 
• 

Another conceptual reference are the Twitch Plays Pokémon streams. In these game streams, a game from the Pokémon franchsie is run, being controlled entirely by commands given by the viewers on the chat. For example, if a person were to say "A+B" in the chat, this would be taken as an in-game command to press the A and B buttons at the same time. Though not a particularly optimal way to play the game, it is an interesting approach as how to create a cooperative multiplayer experience within a single-player game.

Development

Twitter search

The first attempts were to connect the ESP8266 to the internet and use it to access the Twitter API to retrieve tweets without issuing any additional commands. The Twitter API library for Arduino comes with three basic functions:

• Search for a keyword

std::string search_str;
String tmsg = tcr.searchTwitter(search_str);

• Retrieve user information

std::string search_str;
tcr.searchUser(search_str);

• Post to Twitter

std::string twitter_post_msg;
tcr.tweet(twitter_post_msg);

The first function was used in the core of the project, while the second and third were only used for some early testing. In order to use the search feature, it is required to define a search string, which can be formatted to include complex parameters (explained in the Developer Twitter Documentation). This function, however, does not support tweet collection or analysis.

Once I understood how to properly use this library and had successful attempts, I moved on to develop a more complex mechanism. Inspired by the previously mentioned Twitch Plays Pokémon, I devised a device which would collect tweets with any one of the directional words "up", "down", "left" and "right" (excluding retweets, in order to prevent duplicates) and use them as commands for the microcontroller. The interesting thing about these words is that they are extensively used in the English language with other conotations, making the search especially random.

std::string search_str = "up OR down OR left OR right -RT";

As explained in the beginning, this project uses a light matrix in which only one LED is lit up at a time. The program starts with the top-left one and, as tweets are collected, the directional words are used to define what is next one to be lit.

LED matrix

The forementioned matrix has the size of 3x3, and the LEDs are stored in the code as a 2D array. That way, they were able to be lit up by having their indexes called.

const int rows = 3;
const int columns = 3;
uint8_t pinVal[rows][columns] = { { D0, D1, D2 }, { D3, D4, D5 }, { D6, D7, D8 } };

In order to light them up:

digitalWrite(pinVal[a][b], HIGH);

Commanding

The logic used to convert the tweets into commands is illustrated below. The existence or lack of each word in the tweet was checked in order to return a index-changing value.

This logic was translated into the following function (for columns):

//check is the retrieved tweet
int findDirHor (String check) {
  if (check.indexOf("left") != std::string::npos && check.indexOf("right") != std::string::npos) {
    Serial.println("left and right");
    return 0;
  } if (check.indexOf("right") != std::string::npos) {
    Serial.println("right");
    return 1;
  } else if (check.indexOf("left") != std::string::npos) {
    Serial.println("left");
    return -1;
  }
  Serial.println("nor left nor right");
  return 0;
}

However, since the LEDs are declared in a 3x3 array, the indexes mustn't ever exceed 2 or go below 0. Therefore, the function for applying the index change was written taking these cases into consideration.

//val is returned from the findDirHor() function
void columnIndex (int val) {
  if (j == 0 && val == -1) { j = 2;
  } else if (j == 2 && val == 1) {  j = 0;
  } else { j = j + val;
  }}

These functions were adapted and used both for columns and rows and can be expanded into different sized matrixes.

Prototype

The first prototype was built on a breadboard using basic components attached to ESP8266. Below it, there is the list of tweets used to move the LED light.

down
nor left nor right
MSG: "my parents r bout to put my ass up for adoption"

nor up nor down
right
MSG: "@GeneralHospital
 that's right #Elizabeth @RebeccaLHerbst Call 1-800-Jason
@1SteveBurton not the police"

nor up nor down
right
MSG: "@GeneralHospital
 that's right #Elizabeth @RebeccaLHerbst Call 1-800-Jason
@1SteveBurton not the police"

down
nor left nor right
MSG: "I wrote another post (on Warwick's blogging platform)

I do have one coming up on my personal blog. I'm just finalising it and that will hopefully be up by the end of the week!"

down
nor left nor right
MSG: "I wrote another post (on Warwick's blogging platform)

I do have one coming up on my personal blog. I'm just finalising it and that will hopefully be up by the end of the week!"

down
nor left nor right
MSG:"I wrote another post (on Warwick's blogging platform)

I do have one coming up on my personal blog. I'm just finalising it and that will hopefully be up by the end of the week!"

down
nor left nor right
MSG:"I wrote another post (on Warwick's blogging platform)

I do have one coming up on my personal blog. I'm just finalising it and that will hopefully be up by the end of the week!"

nor up nor down
right
MSG: "@Mylesvance09 @DLovesRasslin You’re right; he don’t know bout that life"

nor up nor down
right
MSG: "@Mylesvance09 @DLovesRasslin You’re right; he don’t know bout that life"

This prototype showed that the functions worked as intended, but showed some parts that would need correction. First of all, the delay between each search is quite high (10 seconds) which makes it tedious to watch. Second, there is a big delay between turning off a LED and lighting up the next, besides the 10 seconds gap. Finally, many of the commands were issued repeatedly because of repeated tweets.

By changing the delay between searches, I realized that the ideal delay is actually the default found in the library example: 20 seconds. Setting this value lower makes it more likely that the program will retrieve the same tweet repeatedly. On the other hand, I was able to modify the program structure to minimize the delay between LEDs lighting up. Though the device is not working fast, it is at least visually smooth.

For the next prototype, I want to work with a bigger matrix, and try to make the tweet search smoother and faster. Ideally, I would be able to only issue commands to the microcontroller with the arrival of a new tweet.