CogniMem SDK

Introduction

The CogniMem library allows programmers to integrate into their applications a pattern learning and recognition engine based on a CogniMem neural network. The patterns can derive from sensor data, analog signal, video images, text data, statistics, and more. Any phenomenon which can have a digital signature (such as a sequence of bytes) can be learned and recognized by the network.

Package Contents

CogniMem.dll
Library with functions to initialize the network, learn and classify vectors, save and restore the neuron contents, execute a simple vector broadcast with recognition. Interface to the CogniMem hardware via USB interface, or simulate the neurons via software if the hardware is not detected.

GVImage.dll
This complementary library includes functions to load, save and display images, as well as some feature extrcations in addition to the one built into the CM1K chip.

Midlib2.dll
This undocumented DLL must be copied in the same folder as the CogniMem.dll. It is mandatory to detect and interface with the CM_EB and CM-IR board.

Include files
Cognimem.h (header to access the neurons of the CM1K chips through the USB port of the CM_EB and CM_IR boards)
EB_platform.h (header to access the sensor and flash components of the CM_EB and CM_IR boards)
CM_Advanced.h (header of advanced functions)
GV_Image.h (header of image file management functions)

Lib files
Cognimem.lib (must be linked to your project)

Getting started (Example code)

The following example codes demonstrate how easy it can be to write an application taking advantage of the pattern recognition capabilities of CogniMem.

Starting an application

Configuration #1
CM_EB not use in conjunction with a video sensor
Error= CM_Open(CM_EB) //must be equal to 0 otherwise there is a problem with USB connection
NeuronsAvailable=CM_Init;
Enter into main program

Configuration #2
CM_IR OR CM_EB connected to a Micron sensor board
AND the Micron sensor file stored at a known location FilePath
Error= CM_Open(CM_EB) //must be equal to 0 otherwise there is a problem with USB connection
NeuronsAvailable=CM_Init;
VI_Detect(FilePath, MyImage) //The attributes of the structure MyImage are filled and
//the memory necessary to store the video frames is allocated
Enter into main program

Configuration #3
CM_IR OR CM_EB connected to a Micron sensor board BUT the Micron sensor file is not found
OR
CM_EB connected to a non Micron sensor
Error= CM_Open(CM_EB) //must be equal to 0 otherwise there is a problem with USB connection
NeuronsAvailable=CM_Init;
//Edit the attributes of the MyImage structure and
//Allocate the memory to store the video frame manually
MyImage.Width= wwww;
MyImage.Height=hhhh;
MyImage.BitperBand=pppp;
MyImage.Bands=cccc;
VI_AllocateBuffer(MyImage)
Enter into main program

REMARK: The CM_EB Diagnostics utility generated a sensor_data.ini file in which the attributes of MyImage are saved. Your application can read this file to retrieve these parameters.

Main program initialization

Start a new project
CM_InitProject

Load an existing project from file
NeuronsCommitted = CM_LoadCKF(Filename)

Read the project currently loaded in the neural network
CM_SaveCKF(Temp.ckf)
NeuronsCommitted = CM_LoadCKF(Temp.ckf)

Display the settings of the project
CM_ReadProject MyProject // to retrieve the names of the categories and
//camera settings, if applicable
NeuronsCommitted = CM_Read(CM_NCOUNT)
Region.Left = CM_Read(CM_LEFT)
Region.Top = CM_Read(CM_TOP)
Region.Width = CM_Read(CM_NWIDTH)
Region.Height = CM_Read(CM_NHEIGHT)

Learn a vector received through neural network data bus

The vector is broadcasted to the neurons with a series of Write commands as follows: All components but the last one are written with the register CM_COMP. The last component is written with the register CM_LCOMP. The supervised learning sequence is terminated by writing to the Category register.

For (i = 0; i
CM_Write CM_LCOMP, Vector(i)
CM_Write CM_CAT, Category_to_learn
*NeuronsCommitted = CM_Read(CM_NCOUNT)
*Reading the count of committed neurons can be of interest to know immediately if the vector adds a new prototype to the existing knowledge. If not, it is still possible that learning the vector caused the shrinking of certain neurons’ influence field.

Recognize a vector received through neural network data bus

For (i = 0; i
CM_Write CM_LCOMP, Vector(i)
Distance= CM_Read CM_DIST
*Do
Dist(i) = Distance
Category(i) =CM_Read CM_CAT
While (Distance <> 0xFFFF)
*In this example, the distance and categories of all the firing neurons are read. Reading the response of the first neuron in the list (i.e. best match) can accommodate many applications. This is what the built-in recognition engine executes when it is enabled.

Recognize a vector received through the digital input bus

Hardware interface
Assuming that the recognition is already running (i.e. RSR[0]=1)…
Category=CM_Read (CM_FEATCAT)
Distance= CM-Read (CM_FEATDIST)

Simulation
CM_RecoVector Vector_input, Vector_length
Category=CM_Read (CM_FEATCAT)
Distance= CM-Read (CM_FEATDIST)

There is no specific order to read the FEATDIST and FEATCAT registers since both values are latched by the recognition engine until the next vector is received.

Saving the knowledge built after a learning session
CM_SaveCKF Filename

Change default settings of the neural network
CM_Write CM_GCR, NewCtxt
CM_Write CM_MINIF, Minif
CM_Write CM_MAXIF, Maxif

Learn a vector received through digital input bus

When using the recognition engine, it is recommended to build the knowledge in advance and load it prior to turning the engine ON (i.e. RSR[0]=1). However, it is possible to learn in real-time. Depending on the speed at which your vector data is received on the digital input bus, you may have time to learn vectors without stopping the recognition. Furthermore, you can find out how they are recognized first and then decide if teaching is necessary.

Hardware interface
*CM_Write CM_RSR, 0
CM_Write CM_CAT, Category_to_learn
*CM_Write CM_RSR, 1
*Interrupting the recognition engine is optional and based on your vector feeding cycle.

Simulation
CM_RecoVector Vector_input, Vector_length
CM_Write CM_CAT, Category_to_learn

CogniMem control functions
word CM_Open (word Device)
This function establishes the I2C serial connection with the board. Upon success of its execution, you can read/write the top registers of the board (at the slave address 158) and also the CogniMem chip registers (at the slave address 148).

Device can take the following values:
0= Software simulation of the CogniMem chip
1= CM-IR
2= CM-EB

word CM_Init(void)
This function initializes the data structures of the CogniMem neural network to their default values including the project and category names, the definition of the region of interest and the contents of the neurons. This function returns the number of neurons available in the network.

void CM_Close();
This function closes the system. It is especially useful to ensure proper closing of an application interfacing with a CM-IR board.

void CM_Write(word Register, word Value)
Write a register of the CogniMem engine with Value. These registers are described in a next paragraph.

word CM_Read(word Register)
Read a register of the CogniMem engine. These registers are described in a next paragraph.

CogniMem chip registers

Neuron registers
The table below lists the registers controlling the CogniMem neural network. For a detailed description of the neurons and their behavior, please refer to the CogniMem Reference Guide.

Built-in recognition engine registers

This paragraph describes registers which are useful when the recognition logic of the CogniMem chip is enable. For more information, please refer to the CogniMem Chip Datasheet

CogniMem has a recognition engine which can be used optionally and returns the category of the neuron with the best match within 36 clock cycles after the receipt of a vector data on the digital V_DATA input bus. The distance and category of the first neuron can be read in the registers FEATDIST and FEATCAT before the next rise of the DIST_VAL and CAT_VAL strobes.

Hardware requirements:

Pin RECO_EN =1
A signal is supplied to the digital input bus with the V_CLK, V_FV and V_DATA signals

REMARK1:
If you have time to read CM_CAT after reading CM_FEAT and before the next rise of V_FV, CM_CAT will return the 2nd best category, if any.

REMARK2:
When the recognition is running, the neuron bus is very active. Except for the registers CM_RSR, CM_FEATDIST and CM_FEATCAT, it is recommended to turn the recognition off prior to executing Read/Write commands accessing the neuron registers. If impractical to do so, the commands can be executed after the rise of the CAT_VAL signal and before the next fall of the V_FV signal.

Software requirements:
The receipt of a signal on the digital input bus is simulated by the CM_RecoVector function.

IMPORTANT:
The category returned by the recognition engine is not necessarily the only one possible. In the event that several neurons recognize the input vector, the recognition engine outputs the distance and category of the first neuron with the smallest distance. The UNC_ line or bit 3 of the RSR register indicate if several neurons have recognized the input vector with different categories. The ID_ line or bit 2 of the RSR register indicate if all firing neurons have recognized the same category.

Built-in VIDEO recognition registers

When the built-in recognition engine is intended for use with a video input signal, the following additional registers define the region of interest to monitor in the video frame. In this case, the recognition engine also generates the vector to recognize from a region of interest. This vector is a Subsample of the region. It is calculated inside the chip or generated in software by the functions CM_RecoVideo and CS_GreySubsample.

Hardware requirements:
Pin RECO_EN =1
Pin VI_EN=1
A signal is supplied to the digital input bus with the V_CLK, V_FV, V_LV and V_DATA signals

Software requirements:
The receipt of a video signal on the digital input is simulated by the CM_RecoVideo function.

word Flash_SaveCKF()
Save to Flash memory the registers necessary to execute the current recognition. These registers are automatically restored from the Flash when the hardware is powered up or when bit 1 of the EB_CSR register is set to 1.

Restore CKF from Flash
This routine is initiated by setting Bit 1 of the EB_CSR to 1 (CM_Write(EB_CSR, 0x02)). It expects to read a specific sequence of data in the pages of the Flash memory as described below. This data format is the one used by the Flash_SaveCKF function. If you want to save other data to Flash you can do so page per page using the Flash_Write and Flash_Read functions.

CKF data format in Flash:

Page 0 :
Sequence of four bytes interpreted as (SlaveAddr, Register, Data_High, Data_Low).
The slave address must be greater than 176 because smaller addresses are reserved for the Cypress, Xilinx and other chips mounted on the board.
Each set of four bytes is automatically transmitted by an I2C master
Up to 66 sequences can be stored (264 bytes / 4)

Page 1:
Description of the region of interest if applicable
Sequence of 2 bytes for the following registers
RSR, LEFT, TOP, NWIDTH, NHEIGHT, BWIDTH, BHEIGHT

Page i with i>1 :
Content of a neuron described with the following sequence
2 bytes for the NCR register
256 bytes for the 256 COMP
2 bytes for the AIF register
2 bytes for the CAT register
Up to 2046 neurons can be stored from page 2 to 2048

word Flash_Write(word PageID, byte *Data, word Length)
Write the Data array to flash memory starting at page PageID. Length is the size of the array.

word Flash_Read(word PageID, byte *Data, word Length)
Read the content of the flash memory starting at page PageID and return it into the Data array. Length is the size of the array.

Functions for CM-IR board or CM-EB connected to a video sensor

word VI_Detect(char *MISensorDataPath, IMAGEDESC *Image)
This function looks for a sensor_data file describing a Micron sensor and updated the Image structure accordingly. It also allocates the memory buffer necessary to store the video frames. If the function does not find the appropriate *.sdat file it returns “1” and your remedy to retrieve an image from the sensor is to use the function AllocateBuffer described below.

void VI_AllocateBuffer(IMAGEDESC *Image)
This function allocates a memory buffer to grab the video frames according to the sensor parameters edited in the Image structure such as its X and Y resolution and number of bits per pixel.

long VI_Grab (IMAGEDESC *Image)
Grab a video frame and store it in the buffer pointed by the Image pointer. This function returns the number of bytes received. This image can be displayed or saved using the functions of the GVImage.dll described later in this manual.

void VI_Write(byte SlaveAddr, word Register, word Value)
Write a register of the sensor at the SlaveAddr with Value.
The slave address of the Micron MT9V022 mounted on the CM_IR board is 184 or 0xB8. Its most common registers are described below. For a detailed description of all registers, refer to the Micron manual (Micron MT9V022.pdf).
This function is compatible with any other Micron sensor and also with sensor supporting an i2C protocol with 8-bit register followed by 16-bit data as described in the manual of the CM_EB board.

word VI_Read(byte SlaveAddr, word Register)
Read a register of the sensor at the SlaveAddr.
The slave address of the Micron MT9V022 mounted on the CM_IR board is 184 or 0xB8. Its most common registers are described below. For a detailed description of all registers, refer to the Micron manual (Micron MT9V022.pdf).
This function is compatible with any other Micron sensor and also with sensor supporting an i2C protocol with 8-bit register followed by 16-bit data as described in the manual of the CM_EB board.

Micron MT9V022 Sensor Registers

The table below describes the most common registers of the Micron MT9V0022 CMOS sensor. For a detailed description of all registers, refer to the Micron manual supplied with the hardware.

Complementary Advanced functions

The GVImage.dll is a library of add-on functions for image file management and display.

Data structures

typedef struct
{
byte *ImagePtr;
word Bands;
word Width;
word Height;
word Pitch;
word BitPerBand;
word Reserved[4];
}IMAGEDESC;

typedef struct
{
word RSR;
word Left;
word Top;
word NWidth;
word NHeight;
word BWidth;
word BHeight;
}CM_REGION;

Image file management functions

void PictureBoxToCS
(HDC ImageHDC, HBITMAP ImageHWnd, ImageWidth, ImageHeight, IMAGEDESC *MyImage)

Copy the contents of a picture box to a CogniSight image memory space. ImagehDC and ImageHWnd, ImageWidth and ImageHeight are the properties of the picture box. MyImage is the resulting descriptor of the image allocated by CogniSight. The attributes image width and height are set to the input parameters ImageWidth and ImageHeight. The number of bands is set to 3 and number of bits per pixel is set to 8.

void CSToPictureBox
(IMAGEDESC *MyImage, HDC ImageHDC, HBITMAP ImageHWnd, word PaletteID)

Copy a CogniSight image to picture box. MyImage is the descriptor of an image stored in the CogniSight image memory space. ImagehDC and ImageHWnd are the properties of the destination picture box.

If MyImage is a monochrome image (bands=3 and Bitperpixel=8), its pixel values can be converted into pseudo-color value using a color lookup table pointed by PaletteID. These lookup tables can be defined using the CS_InitPalette or CS_CreatePalette. The default PaletteID 0 is the grey-level palette.

Void RegionToPictureBox
(IMAGEDESC *Image, HDC ImageHDC,HBITMAP ImageHWnd, word CtrX, word CtrY, word Width, word Height, word Angle, word PaletteID);

Copy a region of a CogniSight image to picture box. MyImage is the descriptor of an image stored in the CogniSight image memory space. ImagehDC and ImageHWnd are the properties of the destination picture box. The parameters Left, Top, Width and Height describe the region to display. Angle indicates a rotation to apply to the region prior to display.

If MyImage is a monochrome image (bands=3 and Bitperpixel=8), its pixel values can be converted into pseudo-color value using a color lookup table pointed by PaletteID. These lookup tables can be defined using the CS_InitPalette or CS_CreatePalette. The default PaletteID 0 is the grey-level palette.

WordBMPFileToCS (char *Filename,IMAGEDESC *Image);
Load the contents of a bitmap file to a CogniSight image memory space. MyImage is the resulting descriptor of the image allocated by CogniSight.

WordJPEGFileToCS (char * Filename, IMAGEDESC *Image);
Load the contents of a jpeg file to a CogniSight image memory space. MyImage is the resulting descriptor of the image allocated by CogniSight.

Void CSToBMPFile (IMAGEDESC *Image,char *Filename);
Save a CogniSight image to a bitmap file.

Void CSToJPEGFile (IMAGEDESC *Image,char * Filename, int quality);
Save a CogniSight image to a jpeg file.

void CS_InitPalette(word PaletteID)
Initialize the palette indexed by PaletteID to a grey-scale palette.

void CS_SetPalette(word PaletteID, byte *Red, byte *Green, byte *Blue)
CogniSight let’s you define up to three color-lookup table to display your monochrome image. By default they are all set to a grey-level palette.

PaletteID can range from 0, 1 or 2. Each palette is an array of [256]*[3] integer. Red, Green and Blue are respectively a list of 256 red, green and blue intensities to associate to the input pixel values.
CogniSight is delivered with example palettes saved in text format, so you can review and edit their red, green and blue shades if necessary.

Feature extraction functions

Void CS_ROIBlockfit
(word *RegionWidth, word *RegionHeight, word Color_Enable, word *BlockWidth, word *BlockHeight);
This function is useful to calculate the best block size to fit 256 blocks inside a region of interest. Since the CogniMem neurons can learn patterns of 256 components it is best to fit the closest to 256 blocks in the region of the image to survey.
The function calculates the size of square blocks such that the region with a size Width*Height fits a number of square blocks the closest to 256. This calculation of the square block allows to extract a subsample from the region of interest with the best resolution to fit in the neurons’ 256-bytes memory.

T

authorized length of Vector)

*Vector);

image. The rotation angle is an input parameter of the function.