The era of gesture is upon us. No longer do we need to waste our time with keyboards and mice, we can move our bodies in pre-programmed sequences and the computer will somehow understand us. At least that is how it is made up to be. In reality, the technology isn't quite as good as the science fiction movies portray, but things are improving. The IR Gesture Click board produced by MikroElektronika is a step towards that goal.
The Avago Technologies APDS-9960 which is at the heart of the board isn't just a gesture sensing chip. Although that is one of its features, it can also perform:
- Ambient light readings
- Proximity Sensing
- RGB Color Sensing
For this weeks walk through, we are going to be examining some of the vast features of the chip. I say vast, because the multiples of combinations of settings that are available are quite large. To fully explain each feature we will need much more than an introduction. The source code can be found on our GIT repository or Libstock.
https://github.com/MikroElektronika/Click_IR_Gesture_APDS-9960
The concept
is this, if you have 4 light detectors in a square pattern and you are detecting the light output from the LED below and there isn't any reflective object above the detector, what does it detect? Nothing. But when you pass something over it the IR light would reflect off the object and be detected. If you move your hand over it slowly, one side would start detecting light before the others and here is the clue. If you detect light on the R detector first, you are moving your hand to the left. If you detect it first on the D detector, you're moving your hand from the bottom to the top. How clever.
In a perfect movement of a hand, you would see something like this:
Any answer as to why the Left and Right are equal? Because they detect the same amount of light in a up / down movement. Ideal means the best possible scenario, think robot hand in a lab, with perfect surfaces and balanced lighting. The concept is still sound and with a little software magic, you too can have a robot hand.
Let's code - Initialization
The IR Gesture click requires an I2C or TWI bus to communicate over. The first step is to initialize that bus. The IR Gesture is compatible upto 400kHz so most any MCU will suffice.
Initialization is done through the ir_gesture_init( uint8_t address ); function. The default address for the sensor is 0x39 but it has already been defined in the ir_gesture_hw.h file as #define APDS9960_I2C_ADDR 0x39. An initialization would look like:
#include "ir_gesture_hw.h"
static int system_init()
{
I2C1_Init_Advanced( 400000, &_GPIO_MODULE_I2C1_PB67 );
// Initialize APDS-9960
ir_gesture_init( APDS9960_I2C_ADDR );
return 0;
}
Initialization won't do much for you. The sensor will need to have one of its features enabled. Gesture, light, RGB, Proximity. all have their own function to enable the behavior. For gesture we would need to invoke ir_gesture_enable_gesture_sensor( true ); for proximity we would call ir_gesture_enable_proximity_sensor( true ); and so on. Adding the true or false as an argument enables or disables the interrupt for that feature.
#include "ir_gesture_hw.h"
static int system_init()
{
I2C1_Init_Advanced( 400000, &_GPIO_MODULE_I2C1_PB67 );
// Initialize APDS-9960
ir_gesture_init( APDS9960_I2C_ADDR );
ir_gesture_enable_gesture_sensor( true );
return 0;
}
Interrupts
One critical point of the sensor is how it alerts the application of a gesture or proximity. That is done through interrupts. The "int" pin connected to the host will have a falling edge when some motion passes the threshold. We are using a STM32F107VC for this demo and need to enable the interrupt on pin D10 as well as providing an ISR.
static int system_init()
{
DisableInterrupts();
// Set interrupt pin as input
GPIO_Digital_Input( &GPIOD_BASE, _GPIO_PINMASK_10 );
AFIO_EXTICR3 = 0x0300;
EXTI_FTSR |= ( 1 << TR10 ); // Set interrupt on Rising edge
EXTI_IMR |= ( 1 << MR10 ); // Set mask
NVIC_IntEnable( IVT_INT_EXTI15_10 );// Enable External interrupt
I2C1_Init_Advanced( 400000, &_GPIO_MODULE_I2C1_PB67 );
// Initialize APDS-9960
ir_gesture_init( APDS9960_I2C_ADDR );
ir_gesture_enable_gesture_sensor( true );
EnableInterrupts();
return 0;
}
void gesture_ISR() iv IVT_INT_EXTI15_10 ics ICS_AUTO
{
EXTI_PR |= ( 1 << PR10 );
if( ir_gesture_is_interrupted( INT_GESTURE ) )
{
// Do something with the gesture
}
}
The function ir_gesture_is_interrupted( int ); returns a true if the interrupt you are inquiring about has been triggered. The choices are :
- INT_GESTURE
- INT_AMBIENT_LIGHT
- INT_PROXIMITY
*Note
Optionally, while not a great idea, you can poll the interrupt with the ir_gesture_available(); function. This will return true if a gesture is available to read.
Handling the Gesture
Once the sensor reports that a gesture has occurred, you need to read the gesture direction. That is done by the ir_gesture_read_gesture(); function and returns a gesture_dir_t which is an enumerated type with the following members:
- DIR_NONE
- DIR_LEFT
- DIR_RIGHT
- DIR_UP
- DIR_DOWN
- DIR_NEAR
- DIR_FAR
- DIR_ALL
You could handle the gesture like the following:
static void process_gesture()
{
if ( ir_gesture_available() )
{
switch ( ir_gesture_read_gesture() )
{
case DIR_UP:
// Make coffee
break;
case DIR_DOWN:
// Make eggs
break;
case DIR_LEFT:
// Make toast
break;
case DIR_RIGHT:
// Make bacon
break;
default:
// Ask to try again
}
}
}
Fine Tuning
Light is a funny thing. There is always a changing range of amounts of it and because of the environment that the sensor is in, you may need to change a few things. Some of those are gain, drive, low level thresholds, high level thresholds, turning off directions that you don't intend to use and so forth. Special functions also exist to boost the output of the onboard LED, void ir_gesture_set_led_boost( uint8_t boost ); . By default, some of those parameters are set for you when you enable the feature, however your mileage may vary and some experimentation will be needed to obtain desired results. For example, we need a higher gain on proximity due to strong external lighting, we can increase the proximity sensitivity by 4x:
//Increase gain by 4x ir_gesture_set_proximity_gain( 2 );
Final Notes
The most intriguing function of the Avago Technologies APDS-9960 is the ability to measure gesture and proximity. Other valuable functions exist such as ambient light. One possible use is on display that you wanted to adjust back lighting on the display but also could turn off the display when placed against the face. Color detection is also incredibly useful for color balance. The sensor could balance the RGB display lighting with the input of surrounding lighting. Exciting.
Example
/******************************************************************************* * Title : IR Gesture Example * Filename : MikroC_ARM_M3.c * Author : RBL * Origin Date : 15/01/2016 * Notes : None *******************************************************************************/ /*************** MODULE REVISION LOG ****************************************** * * Date Software Version Initials Description * 15/01/16 .1 RBL Module Created. * *******************************************************************************/ /** * @file MikroC_ARM_M3.c.c * @brief This module contains the */ /****************************************************************************** * Includes *******************************************************************************/ #include#include "ir_gesture_hw.h" #include "resources.h" #include "gesture.h" /****************************************************************************** * Module Preprocessor Constants *******************************************************************************/ #define PROX_INT_HIGH 50 // Proximity level for interrupt #define PROX_INT_LOW 0 // No far interrupt /****************************************************************************** * Module Preprocessor Macros *******************************************************************************/ /****************************************************************************** * Module Typedefs *******************************************************************************/ /****************************************************************************** * Module Variable Definitions *******************************************************************************/ static volatile bool isr_flag; unsigned int TFT_DataPort at GPIOE_ODR; sbit TFT_RST at GPIOE_ODR.B8; sbit TFT_RS at GPIOE_ODR.B12; sbit TFT_CS at GPIOE_ODR.B15; sbit TFT_RD at GPIOE_ODR.B10; sbit TFT_WR at GPIOE_ODR.B11; sbit TFT_BLED at GPIOE_ODR.B9; sbit APDS9960_INT at GPIOD_IDR.B10; #if defined( DEBUG ) sbit Mmc_Chip_Select at GPIOD_ODR.B3; #endif /****************************************************************************** * Function Prototypes *******************************************************************************/ /** * @brief Initialize Display */ static void display_init( void ); /** * @brief Process Incoming Gesture */ static void process_gesture( void ); /** * @brief Initialize System Peripherials */ static int system_init( void ); /****************************************************************************** * Function Definitions *******************************************************************************/ static void display_init() { TFT_Init_ILI9341_8bit( 320, 240 ); TFT_Set_Font( TFT_defaultFont, CL_WHITE, FO_HORIZONTAL ); TFT_Fill_Screen( CL_WHITE ); TFT_Set_Brush(1, CL_WHITE , 0, 0, 0, 0); TFT_Set_Pen( CL_BLUE, 1 ); TFT_Line( 20, 220, 300, 220 ); TFT_LIne( 20, 40, 300, 40 ); TFT_Set_Pen( CL_WHITE, 1 ); TFT_Set_Font( &HandelGothic_BT21x22_Regular, CL_BLUE, FO_HORIZONTAL ); TFT_Write_Text( "IR Gesture click", 90, 10 ); TFT_Set_Font( &Verdana12x13_Regular, CL_BLUE, FO_HORIZONTAL ); TFT_Write_Text( "EasyMx PRO v7 for STM32", 19, 223 ); TFT_Set_Font( &Verdana12x13_Regular, CL_RED, FO_HORIZONTAL ); TFT_Write_Text( "www.mikroe.com", 200, 223 ); TFT_Set_Font( &TFT_defaultFont, CL_RED, FO_HORIZONTAL ); TFT_Rectangle( 80, 60, 300, 200 ); TFT_Write_Text( "Please wait...", 100, 110 ); } static void process_gesture() { if ( ir_gesture_available() ) { TFT_Set_Font( &TFT_defaultFont, CL_RED, FO_HORIZONTAL ); TFT_Rectangle( 80, 60, 300, 180 ); switch ( ir_gesture_read_gesture() ) { case DIR_UP: TFT_Image(110,80, touch_up_bmp, 1); break; case DIR_DOWN: TFT_Image(110,80, touch_down_bmp, 1); break; case DIR_LEFT: TFT_Image(110,80, touch_left_bmp, 1); break; case DIR_RIGHT: TFT_Image(110,80, touch_right_bmp, 1); break; case DIR_NEAR: TFT_Write_Text("NEAR", 100, 110 ); break; case DIR_FAR: TFT_Write_Text("FAR", 100, 110 ); break; default: TFT_Write_Text("NONE", 100, 110 ); } } } static int system_init() { DisableInterrupts(); // Set interrupt pin as input GPIO_Digital_Input( &GPIOD_BASE, _GPIO_PINMASK_10 ); AFIO_EXTICR3 = 0x0300; EXTI_FTSR |= ( 1 << TR10 ); // Set interrupt on Rising edge EXTI_IMR |= ( 1 << MR10 ); // Set mask NVIC_IntEnable( IVT_INT_EXTI15_10 );// Enable External interrupt I2C1_Init_Advanced( 400000, &_GPIO_MODULE_I2C1_PB67 ); delay_ms( 100 ); // Initialize APDS-9960 ir_gesture_init( APDS9960_I2C_ADDR ); ir_gesture_enable_gesture_sensor( true ); display_init(); EnableInterrupts(); return 0; } void main() { if( system_init() ) return; while (1) { if( isr_flag ) { DisableInterrupts(); process_gesture(); isr_flag = false; EnableInterrupts(); } } } void gesture_ISR() iv IVT_INT_EXTI15_10 ics ICS_AUTO { EXTI_PR |= ( 1 << PR10 ); if( ir_gesture_is_interrupted( INT_GESTURE ) ) { isr_flag = true; } }
