Humanoid Based Personal Assistant

348
Final Robot

OBJECTIVE

The main objective of the project is to develop  a humanoid robot that will follow the user where ever he goes and responds to what he asks. It will also detect the user’s face and remember him. There are many modifications that can be made in this project and will be listed down as well.

The main features of the humanoid are listed as under:

žFace detection and recognition

žColor detection based  follower

žVoice controlled command responder.

žFace detection for door security at homes.

RELEVANCE OF THE PROJECT

ž Relevance of the project can be in the following fields:

žSource of entertainment and information for blind/visually impaired.

ž Voice based calculator can be used to teach visually impaired students or it can be a game for visually sound students.

ž Security systems can use the face recognition and similarity checking.

HARDWARE USED

  • Raspberry Pi
  • Camera Module
  • L293D Motor Driver
  • Robot Chassis
  • Wireless Speaker
  • Soundcard
  • Microphone
  • DVI Cable
  • DC Motor
  • Servo Motor
  • Power Supply for motors

SOFTWARES USED

—RASPBIAN– Raspbian is a Debian-based computer operating system for Raspberry Pi. Since 2015 it has been officially provided by the Raspberry Pi Foundation as the primary operating system for the family of Raspberry Pi single-board computers.It is an open source and free software.

—PYTHON-Python is a widely used high-level programming language for general-purpose programming, created by Guido van Rossum and first released in 1991. An interpreted language, Python has a design philosophy that emphasizes code readability , and a syntax that allows programmers to express concepts in fewer lines of code than might be used in languages such as C++ or Java.

—OPENCV OpenCV (Open Source Computer Vision) is a library of programming functions mainly aimed at real-time computer vision. Originally developed by Intel, it was later supported by Willow Garage and is now maintained by Itseez. The library is cross-platform and free for use under the open-source BSD license.

CONCEPT OF COLOR DETECTION

  • Colors on a computer are represented in what is called a color-space. Color-space is also known as color models, which describes the range of colors as tuples of numbers. We won’t go over every color-space but we will focus on two main ones, BGR (Blue, Green, Red) and HSV (Hue, Saturation, Value).
  • In a BGR color-space, there are three parameters Blue, Green and Red. Each of these values typically goes from 0-255. For example, if we were to show a pure green pixel on-screen, then the B value would be 0, the G value would be 255, and the R value would be 0. This makes sense as there are no blue or red in a pure green pixel. Also the order you put these number in makes a difference. B is always first, then G, then R.
  • However, in HSV color space, the three parameters are Hue, Saturation, and Value. In simplest term Hue is the color, Saturation is how intense the color is and Value is the brightness of the color. The cylinder shows it graphically.

RESULT OF COLOR DETECTION

COLOR DETECTION RESULT

CONCEPT OF FACE DETECTION

—Object Detection using Haar feature-based cascade classifiers is an effective object detection method proposed by Paul Viola and Michael Jones in their paper, “Rapid Object Detection using a Boosted Cascade of Simple Features” in 2001. It is a machine learning based approach where a cascade function is trained from a lot of positive and negative images. It is then used to detect objects in other images.

—HAAR-CASCADE DETECTION IN OpenCV

—OpenCV comes with a trainer as well as detector. If you want to train your own classifier for any object like car, planes etc. you can use OpenCV to create one.

—Here we will deal with detection. OpenCV already contains many pre-trained classifiers for face, eyes, smile etc. Those XML files are stored in opencv/data/haarcascades/ folder. Let’s create face and eye detector with OpenCV.

—First we need to load the required XML classifiers. Then load our input image (or video) in grayscale mode.

RESULT OF FACE DETECTION

FACE DETECTION RESULT

The picamera is used to perform face detection. It can detect the number of faces in front of the camera and store it in its memory. This result it can be used for various security purposes. The  image is the result we got when we performed face detection using this project.

CONCEPT OF VOICE CONTROLLED COMMAND RESPONDER

—Alexa is an intelligent personal assistant developed by Amazon, first used in the Amazon Echo and the Amazon Echo Dot devices developed by Amazon Lab126. It is capable of voice interaction, music playback, making to-do lists, setting alarms, streaming podcasts, playing audiobooks, and providing weather, traffic, and other real time information, such as news. Alexa can also control several smart devices using itself as a home automation system.

—Most devices with Alexa allow users to activate the device using a wake-word (such as Echo); other devices (such as the Amazon app on iOS or Android) require the user to push a button to activate Alexa’s listening mode. Currently, interaction and communication with Alexa is only available in English and German. However, support for Hindi and Japanese is rumored for launch in late 2017. In November 2017, Alexa became available in the Canadian market in English only.

FINAL ROBOT PRODUCT

Final Robot

FINAL ROBOT PRODUCT

Robot Top View

ROBOT TOP VIEW

Robot Back View

ROBOT BACK VIEW

FUTURE SCOPE

—The humanoid can be more precisely improved in its movements i.e hand and leg movements as actual humans.

—For the initial stages of development only a single pi camera is being used that is fixed but the cameras could be doubled and made moving i.e they could be rotated 180 each if a small servo motor is placed behind them.

—A client to client connection could be made between two nodemcu modules to make the home automation part wireless.

—Ultrasonic sensor could be added to make it object avoidable.

—Although , face detection has already been specified about here but we personally feel that the robot should speak out the name of the person once it recognizes him/her .

—Some more modifications such as pick and drop/ pick and place features could be used and a fully functional robotic arm concept could be combined along with this.

—Alexa trigger could be used for sending and receiving e-mails, sms , playing music on phone and other such features which will enhance the quality of the personal assistant.

RELEVANCE AND PROJECT HIGHLIGHTS

— Relevance of the project can be in the following fields:
• Source of entertainment and information for blind/visually impaired.
• Voice based calculator can be used to teach visually impaired students or it can be a game for visually sound students.
• Security systems can use the face recognition and similarity checking.

—Project Highlights : This project can act as a prototype for many advanced applications. Usually projects belong to a single industry in terms of application, but this project has various aspects like entertainment, computation, face recognition and security. It can help the visually impaired to connect with the world by giving them access to Wikipedia, Calculator , weather updates all through their voice. The project can also keep people secure as it can be used as a surveillance system which captures the face of the person standing at the door.

To know more about the uses of Raspberry Pi refer to the link below :

Uses of Raspberry Pi

1 COMMENT

LEAVE A REPLY

Please enter your comment!
Please enter your name here