Langsung ke konten utama

CV :: Detect and Track Objects in Live Webcam Video Based on Color and Size Using C#

Introduction

You can select a color in real time and it tracks that color object and gives you the position. I use the Aforge library for that. I also used .NET Framework 4.0. It is a C# desktop application, it can take up to 25 frames per second. You can change color size any time you want, the color of drawing point will also change.

Background 

I saw a very interesting project in CodeProject named Making of Lego pit camera. With the help of this project, I thought a real time tracker could be made where the color and object's size could also be changed in real time. and can draw the movement of that object in a bitmap with that color. I used some part of their code, although I used a separate color filter for more accuracy.

Using the Code 

The steps are very simple:
  1. Take videoframe from webcam
  2. Use filtering by given color (here Euclidian filtering is used)  
  3. Make greyscale
  4. Find objects by given size
  5. Find biggest object
  6. Draw object position in bitmap
First, I will explain how my software works. It can start tracking by the default color black and a color range of 120, but you can change that in real time. To change color, use the color dialog to select a color, remember to press OK and check in the solid color box if your required color is present or not.
To select the range and define the minimum size of the object, see the next image, there are three numeric updown lists for that. objectsize is calculated in pixels. Now about the views:
There are four views, three of them are Aforge video source controls and the other is a PictureBox. The first one shows a normal video, the second one shows all the detected objects, the third box shows only the biggest object, and the fourth one just draws a 0 sign in the biggest object location. See the below image for a clearer view:
I also added a RichTextBox which shows the pixel of the biggestrectangle position. Now about the code for connection, I use normal Aforge connection code. 
For that, you need to download the Aforge DLL, and also add a very good video control for showing video. You can add it to your toolbox. Just click a free space on toolbox, choose items-> browse the DLL -> add the control.
using AForge;
using AForge.Video.DirectShow;
Then initialize:
videoDevices = new FilterInfoCollection(FilterCategory.VideoInputDevice);
if (videoDevices.Count == 0)
    throw new ApplicationException();
foreach (FilterInfo device in videoDevices)
{
    VideoCaptureDevice videoSource = new VideoCaptureDevice(device.MonikerString);
    videoSource.DesiredFrameSize = new Size(320, 240);
    videoSource.DesiredFrameRate = 15;
    videoSourcePlayer1.VideoSource = videoSource;
    videoSourcePlayer1.Start();
}
For videoplayercontrol, add an event in its misc named:
 private void videoSourcePlayer1_NewFrame(object sender, ref Bitmap image)
{
//work with image
}
For understanding how filtering works, I will explain using a picture of a jellyfish. Here, I take a red center color for understanding. In my provided software, the user can choose his requested color and size. 
In that function, I apply my color detection code by writing Euclidean filtering and then use blob counter to extract data. I will explain it in detail here. 
To see EuclideanColorFiltering work, first see:
undefined
We are going to apply a color filter. It is a very simple code to use euclideanfiltering:
// create filter
EuclideanColorFiltering filter = new EuclideanColorFiltering( );
// set center colol and radius
filter.CenterColor = Color.FromArgb( 215, 30, 30 );
filter.Radius = 100;
// apply the filter
filter.ApplyInPlace( image ); 
Now see the effect:
Well, to understand how it works, look closely at the code: 
filter.CenterColor = Color.FromArgb( 215, 30, 30 );
filter.Radius = 100
The first line selects the selected color value. You all know that color has a value 0 to 255. Using filter.CenterColor = Color.FromArgb( 215, 30, 30 ); I specify my center color will be a red effected color because here the value of red is 215, green and blue is 30, and filter.Radius = 100 means that all color values near 100 are in my specified color. Now my filter filters the pixels, and decides which color is inside/outside of RGB sphere with the specified center and radius – it keeps pixels with colors inside/outside of the specified sphere and fills the rest with the specified color. 
Now for detecting objects, I use bitmap data and use the lockbits method. To clearly understand this method, see here. Then, we make it a greyscale algorithom, then unlock it.
BitmapData objectsData = image.LockBits(new Rectangle(0, 0, 
   image.Width, image.Height),ImageLockMode.ReadOnly, image.PixelFormat);
// grayscaling
UnmanagedImage grayImage =  grayscaleFilter.Apply(new UnmanagedImage(objectsData));
// unlock image
image.UnlockBits(objectsData); 
Now for the object, we use a blobcounter. It is a very strong class that Aforge provides.
blobCounter.MinWidth = 5;
blobCounter.MinHeight = 5;
blobCounter.FilterBlobs = true;
blobCounter.ProcessImage(grayImage);
Rectangle[] rects = blobCounter.GetObjectRectangles();
foreach(Rectangle recs in rects)
if (rects.Length > 0)
{
    foreach (Rectangle objectRect in rects)
    {
        Graphics g = Graphics.FromImage(image);
        using (Pen pen = new Pen(Color.FromArgb(160, 255, 160), 5))
        {
            g.DrawRectangle(pen, objectRect);
        }

        g.Dispose();
        }
    }
blobCounter.MinWidth and blobCounter.MinHeight define the smallest size of the object in pixels, and blobCounter.GetObjectRectangles() returns all the objects rectangle position, and using the graphicsclass, I draw a rectangle over the images.
Now, if you want to take only the biggest object, there is a method for that:
blobCounter.MinWidth = 5;
blobCounter.MinHeight = 5;
blobCounter.FilterBlobs = true;
blobCounter.ObjectsOrder = ObjectsOrder.Size;
blobCounter.ProcessImage(grayImage);
Rectangle[] rects = blobCounter.GetObjectRectangles();
foreach(Rectangle recs in rects)
    if (rects.Length > 0)
    {
       Rectangle objectRect = rects[0];
       Graphics g = Graphics.FromImage(image);
       using (Pen pen = new Pen(Color.FromArgb(160, 255, 160), 5))
       {
           g.DrawRectangle(pen, objectRect);
       }
       g.Dispose();
    }
Now the image is like this:
But if you want to extract, you can use the following code:
Bitmap bmp = new Bitmap(rects[0].Width, rects[0].Height);
Graphics g = Graphics.FromImage(bmp);
g.DrawImage(c, 0, 0, rects[0], GraphicsUnit.Pixel);
So you will get your object like that:
undefined
For drawing the bitmap, I had to use threading. The thread was called in the videosurecplayer event.
void p(object r)
{
   try
   {//create image
   Bitmap b = new Bitmap(pictureBox1.Image);
   Rectangle a = (Rectangle)r;
   Pen pen1 = new Pen(Color.FromArgb(160, 255, 160), 3);
   Graphics g2 = Graphics.FromImage(b);
   pen1 = new Pen(color, 3);
   SolidBrush b5 = new SolidBrush(color);
    Font f = new Font(Font, FontStyle.Bold);
   g2.DrawString("o", f, b5, a.Location);//draw the position
   g2.Dispose();
   pictureBox1.Image = (System.Drawing.Image)b;
   this.Invoke((MethodInvoker)delegate
       {
           richTextBox1.Text = a.Location.ToString() + 
   "\n" + richTextBox1.Text + "\n"; ;
       });
   }
   catch (Exception faa)
   {
       Thread.CurrentThread.Abort();
   }
  Thread.CurrentThread.Abort();
}
I have to use invoke to write in the textbox because of cross threading. To send the rectangle object, I use a parameterized thread. The thread aa calls the function p() to draw on the bitmap and write in the textbox.
[__strong__]if (rects.Length > 0)
{
    Rectangle objectRect = rects[0];

    // draw rectangle around derected object
    Graphics g = Graphics.FromImage(image);

    using (Pen pen = new Pen(Color.FromArgb(160, 255, 160), 5))
    {
        g.DrawRectangle(pen, objectRect);
    }
    g.Dispose();
    int objectX = objectRect.X + objectRect.Width / 2 - image.Width / 2;
    int objectY = image.Height / 2 - (objectRect.Y + objectRect.Height / 2);
    ParameterizedThreadStart t = new ParameterizedThreadStart(p);
    Thread aa = new Thread(t);
    aa.Start(rects[0]);
}

Komentar

Postingan populer dari blog ini

CV :: Simple Color Tracking Menggunakan Webcam Dengan Library AForge.NET

Simple Color Tracking Menggunakan Webcam Dengan Library AForge.NET Berikut installer demo software simple color tracking. http://www.mediafire.com/file/304priwevhameq0/simple%20color%20tracking.rar Schematic dan firmware http://www.mediafire.com/file/mqdrsjj3qoc7777/Color_Tracking-wangready.wordpress.com.rar Beberapa waktu lalu saat sempat terpikir untuk membuat aplikasi image processing, saya menemukan sebuah library yang saya kira cukup simple untuk diimplementasikan yaitu  AForge.NET  untuk bahasa C#. Alhamdulillah saat itu ada beberapa perangkat yang tersedia di laboratorium sehigga bisa terealisasi. Berikut uraian saya. Simple Color Tracking Abstraksi Robotics vision adalah salah satu bidang kajian dalam dunia robotika. Salah satu langkah awal untuk memulainya adalah robot mampu mengenali warna. Dalam kesempatan kali ini, robot didisain mampu mengenali warna lalu mengikuti gerak dari warna yang terdeteksi tersebut. Sebagai sensornya digunakan ...

ARDUINO :: Komunikasi Arduino Uno dengan Komputer

Komunikasi serial Arduino  adalah Komunikasi antara Arduino Uno dan Komputer dapat dilakukan melalui port serial (via USB). Dalam hal ini, Arduino Uno tidak hanya bisa membaca data dari komputer yang ada di port serial, melainkan juga dapat mengirim data ke komputer. Sehingga komunikasi yang dilakukan bersifat dua arah. Pada Arduino IDE menyesuaikan fasilitas untuk melakukan komunikasi dua arah tersebut melalui serial monitor. Dengan menggunakan fasilitas ini, dapat dikirimkan data ke Arduino Uno dan sebaliknya dapat membaca kiriman dari arduino uno. Tentu saja, hal ini memungkinkan dapat mengontrol Arduino Uno melalui komputer dan memantau sesuatu yang sedang terjadi di Arduino Uno. Sebagai contoh, saat mengirimkan isyarat untuk menghidupkan lampu atau memantau suhu yang terdeteksi oleh sensor suhu di Serial Monitor. ilustrasi sederhana komunikasi antara laptop dengan arduino Jenis Perintah Komukasi serial Arduino Serial.begin() : berguna untuk menentukan kecepatan p...