Projects:$|
A quick look at a few
More details can be found on my gitHub
/Thesis-Project:$|
Details
This project implemented Multi-Cascaded Neural Networks for leaf recognition on an ESP32 Micro controller. The two main stages of this project included re-purposing a facial recognition model to detect plant leaves and then implement and inference onto a micro controller. The project aims to help conserve cloud computing resources by moving the inference onto the edge in a smart farming environment.
This project was the basis of my Thesis, which I will happily send upon request. A warning though, it is 50 pages long.
the process in its simplest form goes like this:
1. pick a small-ish model ( I would probably suggest some version of mobilenets or mobilenetsV2) but ESP-Leaf is MTCNN
more on this model (and how it pertains to ESP-Leaf) is found here: MTCNN-Leaf
train the model however you like ( I used caffe but the process is framework independent, you just need to know the data layout)
2. extract the model's weights and save them into a usable format (text, in the caffe example it gave me a numpy array which was nice)
3. transform the models weights into the ESP-Face library layout (see translateWeights2ESP_lib.py)
**SETUP FOR CAFFE, just do a quick check what your framework uses and transpose accordingly.
This also creates a C header file that you'll use for inference, as a necessary feature you can cut the weights up into however many sections you need. (that was a bad explanation, what i mean is you won't have enough memory to load all your weights in at once, so separate functions are created to call upon the weights that you want to load at a given point, this is best illustrated in the R-Net section of the ESP-Leaf)
4. Finally! all that preparation work is done...now to implement your chosen model onto the ESP-32. Put all the generated C Header files into your project folder so you can use them in your code. Implement your chosen model as you see fit in C, and always test for heap failures.
some extra notes:
check the closed issues section of ESP-Face for conversations of problems (i closed them but they might be useful)
split convolutions where you need or your model won't fit
make sure the raw text weights and the libraries your'e using all fit onto the ESP ( for O-Net the weights would compile and upload just fine but the program would never start)
use heap tracing
/Robotics:$|
Details
Designed with the R4 Astromech droid in mind, but getting lost along the way resulted in R4-04, the droid for a galaxy not so far away.
This droid hosts its own WiFi network, and anyone in reach can accesss a web server holding a livestream of the IP camera and controls for the droid.
The IP of the droid's control server is printed on the bright blue LCD screen.
The droid is controlled using websocket communication on the ESP32 and relays information from the user to an Arduino Mega controlling the motors via I2C commands. The Arduino also handles a self balancing routine, adjusting its shoulders with PID so that rougher terrain won't be a problem.
Along with the motor control, R4-04 also shows information concerning its environment on a second screen. The information is obtained via a BME280 Sensor.
/5 String Banjo:
Details:
This is a full sized 5 string Open Back Banjo I decided to make. It uses nylong strings, planetary geared tuners, a calfskin head and is Texas themed.
/Electric Phin (พิณ):
Details:
This is the model for a full sized 3 string electric Phin, a 3 String Thai lute. Making this instrument required both CNC and 3D printed parts, as well as some hand tools for cleaning things up.
/GreenEnergy:$|
Details
I had come into posession of a trashed ceiling fan, what to do??
Of course, turn it into a wind turbine!
Here is the result of not having a welder and transforming a solar panel control into a Wind Turbine control.
/Wearable Tech:$|
Details
My Xiaomi smartwatch died after a few years of good service oh no =[
What to do? Create a new one! After seeing the HPDL-1414 I knew I needed one on my wrist.
This is a 'Retro' ESP32 Based Smart watch using an HPDL-1414 and an .91'' OLED
It has an ESP32 Wrover-B at its core, an MPU9250, Analog MEMS Microphone,
a Lipo Charging circuit, 3 Buttons, USB-C, and 2 screens!
/Smart Farming:$|
Details
This is a smart farming implementation I made for a small scale farm in Texas.
The system is implemented using a server, database, various data loggers, and water control through multiple communication methods.
/Smart Hospitals:$|
Details
This project aimed primarily to test student likelihood of using a small IoT device to
detect fever.
This project used a Raspberry pi with a node.js server to show users a heat image
obtained from an infrared camera connected to the microprocessor.
Students were asked to fill out a
questionnaire also held on the web app and the results were statistically processed in python.
The
project aimed at creating a way to minimize procedures normally carried out by nurses during
patient check-ins or other medical environments.
/RadioJSON:$|
Details
A fixed wing drone using only an Arduino nano!
This is a custom implementation of an arduino nano as a flight controller along with a custom built transmitter.
I wanted to build a fixed wing drone using some spare parts I had laying around and this is the result! After a lot of testing and soldering, this is the result.