12.5: Closest or Highest Point Tracking – Kinect and Processing Tutorial

[ad_1]
This video looks at an algorithm for finding a particular pixel that is the closest or highest. The idea is to set a “record” value and check each and every pixel to see if they beat the record leaving yourself with the “world record holder” at the end of the for loops. This can be used to track the top a person’s body or even the tip of a finger rather accurately (but only if the person is pointing the finger up!)

Open Kinect links:

Contact:

Learn Processing from scratch:

More about image processing in Processing:

Help us caption & translate this video!


Posted

in

by

Tags:

Comments

24 responses to “12.5: Closest or Highest Point Tracking – Kinect and Processing Tutorial”

  1. Fabio Vaccaro Avatar

    This playlist is wonderful! You should improve it making new videos!

  2. Santy Gallegos Avatar

    You're simply great!!! I've loved working with the Kivect v2 since I've got to your videos!! Hope to have more of this series soon!!

  3. Chris Trottier Avatar

    Are there any plans to extend the Kinect series? I've been loving working through this set.

  4. sama-e-shan shoumo Avatar

    Hello .I am a bit of a newbie.I have a windows 10 machine .so i used processing kinect library for windows by brian chung.I wanted to use the kinect(version 1…using it with kinect SDK1.8) with openCV library.I used the following code…I seem to load the image with no problem but it doesn't detect face can u solve it please:

    import kinect4WinSDK.Kinect;
    import gab.opencv.*;
    import java.awt.Rectangle;

    OpenCV opencv;
    Rectangle[] faces;

    Kinect kinect;

    void setup() {
    size(640, 480);
    opencv = new OpenCV(this, 640/2, 480/2);
    // size(opencv.width, opencv.height);

    opencv.loadCascade(OpenCV.CASCADE_FRONTALFACE);
    faces = opencv.detect();
    kinect = new Kinect(this);
    }

    void draw() {

    PImage img=kinect.GetImage();
    opencv.loadImage(img);
    image(img, 0, 0);

    noFill();
    stroke(0, 255, 0);
    strokeWeight(3);
    for (int i = 0; i < faces.length; i++) {
    rect(faces[i].x, faces[i].y, faces[i].width, faces[i].height);
    }
    }

  5. Daniel Amaya Avatar

    Hi Daniel! What's new with OpenNI? I would love to see some skeleton tracking and gesture recognition in your channel (if possible). Anyway, this kinect playlist is super nice!

  6. Too Good Mate Avatar

    Hi mate! Did you end up trying to play with the Microsoft SDK on a windows machine? I'd like to learn how to incorporate some of the SDK features into my system.

  7. brixius111 Avatar

    The highest pixel example should be able to be optimized by breaking out of the loop once you have found a pixel that qualifies. since your scanning the image from top to bottom. Once y > ry you are lower than the record.

  8. julie bassinot Avatar

    hello hello +The Coding Train ! this channel is wonderful! thk you! im trying to use this for a project and i really need to see if there is more of the code. Where did you write the code? the github link in comments doesn't work 🙁

  9. realcygnus Avatar

    truly fabulous content

  10. Kyle Overton Avatar

    I owe you a dinner the next time I'm in NY. Your channel is amazing; I've learned so much in such a short time. You're an absolutely fantastic educator!

  11. David Stephen Avatar

    How we can turn this to multi hand tracking?
    I saw in one of the comments that you are working on this project.

  12. mediartists Avatar

    Thank you great lesson ^^

  13. Small Details Avatar

    Hi Dan,
    is it possible to get a fullscreen image, even if its stretched with Kinect 1414 video size max only being (640, 480). Been searching for a while now and really exited to show off the finished piece.
    I have my finished piece thanks to you, thoroughly enjoyed your tutorials on this series will get stuck into more projects.

  14. Fiatty Panich Avatar

    @Daniel Shiffman

    hi daniel, this is such a wonderful tutorial.
    but could u give me suggestion if i want to substitute avgX and argy in this interactive mouse example

    int sizeX;
    int sizeY;
    fairy[] fairies = new fairy[200]; //an array of fairy objects
    boolean drawingNet = false; //becomes true when the user clicks
    net activeNet;
    int totalHits = 0;
    int mode = 0;//changes the mode (base colour of the fairies)

    PImage bg; //background image with tree silhouettes
    boolean runAway = true; //if true they are running from the mouse, if false they are running to the mouse

    void setup() {

    sizeX= 500;
    sizeY= 500;
    bg = loadImage("Background.jpg");
    size(500, 500);

    smooth();
    //fills the array with fairies created at random positions
    for (int i =0; i<200; i++) {
    fairies[i] = new fairy(random(0, sizeX), random(0, sizeY), random(-1.5, 1.5), random(-1.5, 1.5), random(2, 8), random(15, 25), random(0, 20));
    }
    }
    void stop()
    {
    //song.close();
    //minim.stop();
    super.stop();
    }

    void draw() {
    //background(bg);
    background(0);
    tint(255, 150);
    //image(bg, 0, 0, sizeX, sizeY);
    noTint();
    noCursor();
    //call react
    //check if user is drawing circle
    for (fairy each : fairies) {
    each.react();
    each.runToOrFrom(mouseX, mouseY);
    if (each.trapped) {
    totalHits += each.hitCount;
    }
    }
    if (!(activeNet==null)) {
    activeNet.drawNet();
    if (totalHits>7000) {
    activeNet.breakNet();
    }
    }
    }

    void mousePressed() {
    //changes the runAway mode when the right button is clicked.
    if (mouseButton == RIGHT) {
    if (runAway) {
    runAway = false;
    }
    else {
    runAway = true;
    }
    }
    else {
    if (activeNet !=null) {
    activeNet.breakNet();
    }
    drawingNet = true;
    activeNet = new net(mouseX, mouseY);
    }
    }

    void mouseReleased() {
    drawingNet = false;
    if (activeNet != null && activeNet.closed==false) {
    activeNet.closeNet();
    for (fairy check : fairies) {
    if (dist(check.xpos+check.xstep, check.ypos+check.ystep, activeNet.centerX, activeNet.centerY)<activeNet.rad/2) {
    check.trapped =true;
    }
    }
    }
    }

    void keyPressed() {
    if (key== ' ') {
    if (mode==0) {
    mode++;
    for (fairy change : fairies) {
    change.baseR=210;
    change.r=210;
    change.baseG=100;
    change.g=100;
    change.baseBLU=255;
    change.blu=255;
    }
    }
    else if (mode==1) {
    mode++;
    for (fairy change : fairies) {
    change.baseR=255;
    change.r=255;
    change.baseG=191;
    change.g=191;
    change.baseBLU=26;
    change.blu=26;
    }
    }
    else {
    mode=0;
    for (fairy change : fairies) {
    change.baseR=120;
    change.r=120;
    change.baseG=191;
    change.g=191;
    change.baseBLU=255;
    change.blu=255;
    }
    }
    }
    }

    //———————————————————–//
    class net {
    boolean closed;

    float centerX;
    float centerY;

    float rad;

    //constructor
    net(float x, float y) {
    centerX = x;
    centerY = y;
    closed = false;
    }

    void drawNet() {
    if (closed) {
    stroke(0);
    }
    else {
    stroke(190, 191, 255);
    }

    strokeWeight(3);
    noFill();
    if (closed) {
    ellipse(centerX, centerY, rad, rad);
    }
    else {
    float temp = dist(centerX, centerY, mouseX, mouseY);
    ellipse(centerX, centerY, 2*temp, 2*temp);
    }
    }

    void closeNet() {
    closed = true;
    drawingNet=false;
    if (mouseX>centerX) {
    rad=2*(dist(centerX, centerY, mouseX, mouseY));
    }
    else {
    rad=centerX-mouseX;
    }
    }

    void breakNet() {
    for (fairy check : fairies) {
    if (check.trapped) {
    check.unTrap();
    }
    }
    totalHits = 0;
    activeNet = null;
    }
    }

    //———————————————————–//
    class fairy {
    boolean trapped = false;
    float xpos;
    float ypos;

    float dx;
    float dy;

    float xstep;
    float ystep;

    float holdX;
    float holdY;

    float breath;//the count that goes up
    float b =0.01;//how much the count goes up by
    float a;//the opacity

    float baseR = 120;//the r component of the original colour
    float r = 120;//r component of the colour
    float g = 191; //g component
    float baseG = 191;//the g component of the original colour
    float blu = 255; //b component
    float baseBLU = 255;//the r component of the original colour

    //the dimensions of the circles
    float inner;
    float outer;
    //hold the original size of the fairy
    float holdIN;
    float holdOUT;

    int hitCount = 0;

    //constructor,
    //gets passed the initial x and y position of the fairy
    //the initial speed and size
    //where the transparency starts
    fairy(float x, float y, float x2, float y2, float x3, float y3, float i) {

    xpos = x;
    ypos = y;

    xstep = x2;
    ystep = y2;

    holdX = x2;
    holdY = y2;

    inner = x3;
    outer = y3;

    holdIN = x3;
    holdOUT = y3;

    breath = i;
    }

    //this method is called in draw()
    //it makes the fairy move, breath, stay within bounds
    void react() {
    //if the mouse is in any of the four corners
    if ((mouseX<sizeX*0.1 &&mouseY<sizeX*0.1)||(mouseX>sizeX*0.9&&mouseY>sizeX*0.9)||(mouseX>sizeX*0.9&&mouseY<sizeX*0.1)||(mouseX<sizeX*0.1&&mouseY>sizeX*0.9)) {
    b=0.2;
    }
    else {
    b=0.01;
    }
    breathe();
    outOfBounds();
    if (trapped) {
    b= b+0.002;
    xstep = xstep*1.01;
    ystep = ystep*1.01;
    }
    xpos = xpos+xstep;
    ypos = ypos+ystep;

    fill(r, g, blu);
    noStroke();
    ellipse(xpos, ypos, inner, inner);

    //if the fairies are running to the mouse, they are brighter the closer they get to it
    if (runAway==false) {
    if (dist(mouseX, mouseY, xpos, ypos)<100) {
    fill(r, g, blu, a*3.5);
    }
    else if (dist(mouseX, mouseY, xpos, ypos)<150) {
    fill(r, g, blu, a*2);
    }
    else {
    fill(r, g, blu, a);
    }
    }
    //if the fairies are running away from the mouse they are brighter the further they get from the mouse
    if (runAway==true) {
    if (dist(mouseX, mouseY, xpos, ypos)>250) {
    fill(r, g, blu, a*3.5);
    }
    else if (dist(mouseX, mouseY, xpos, ypos)>150) {
    fill(r, g, blu, a*2);
    }
    else {
    fill(r, g, blu, a*0.2);
    }
    }
    ellipse(xpos, ypos, outer, outer);
    if (trapped==false) {
    if (inner>holdIN ) {
    inner -= 0.03;
    }
    if (outer>holdOUT) {
    outer-= 0.03;
    }
    //if any of the colours are different to the original colour slowly restore to original state
    if (r>baseR) {
    r-=0.04;
    }
    else if (r<baseR) {
    r+=0.04;
    }
    if (g<baseG) {
    g+=0.05;
    }
    else if (g>baseG) {
    g-=0.05;
    }
    if (blu<baseBLU) {
    blu+=0.05;
    }
    else if (blu>baseBLU) {
    blu-=0.05;
    }
    }
    }

    //if the fairy is about to go offscreen or run in to the edge
    //of a net it turns it around.
    void outOfBounds() {
    if (trapped) {
    //makes sure the fairy doesn't go outside the net
    if (dist(xpos+xstep, ypos+ystep, activeNet.centerX, activeNet.centerY)>activeNet.rad/2) {
    xstep = xstep*-1;
    ystep = ystep*-1;
    hitCount++;
    }
    }
    else {

    if (xpos+xstep<0 || xpos+xstep>sizeX) {
    xstep = xstep*-1;
    }
    if (ypos+ystep<0 || ypos+ystep>sizeY) {
    ystep = ystep*-1;
    }
    if (!(activeNet==null)) {
    if (dist(xpos+xstep, ypos+ystep, activeNet.centerX, activeNet.centerY)<activeNet.rad/2) {
    ystep = ystep*-1;
    xstep = xstep*-1;
    }
    }
    }
    //this makes sure the fairy isn't ever going in a vertical or horizontal line
    if (xstep>-0.5&& xstep<=0 && ystep>0.7) {
    xstep=-0.5;
    }
    else if (xstep<0.5&& xstep>=0 &&ystep>0.7) {
    xstep=0.5;
    }
    if (ystep>-0.5&& ystep<=0 &&xstep>0.7) {
    xstep=-0.5;
    }
    else if (ystep<0.5&& ystep>=0 &&xstep>0.7) {
    xstep=0.5;
    }
    }

    //unTrap is called to reset all of the things that got changed when it was trapped
    void unTrap() {
    trapped = false;
    xstep = holdX;
    ystep = holdY;
    b = 0.01;
    inner += random(5, 10);
    outer += random(5, 20);
    hitCount = 0;
    r +=random(20, 50);
    g-=random(15, 30);
    blu-=random(10, 20);
    }

    //the glow around the fairy fades in and out
    //to look like the fairy is breathing
    void breathe() {
    a = sin(breath)*100;
    //uses a sin wave to change the transparency smoothly;
    breath= breath+b;
    }

    void runToOrFrom(float xp, float yp) {
    dx = 0;
    dy = 0;
    if (dist(xpos, ypos, mouseX, mouseY)<60) {
    dx = xpos – xp;
    dy = ypos – yp;
    if (runAway==false) {
    xpos -=dx/10;
    ypos -=dy/10;
    }
    else if (runAway==true) {
    xpos +=dx/10;
    ypos +=dy/10;
    }
    }
    }
    }

    i have no idea of how to put them in "class" which locate out of void draw()

    it's gonna be delighted and honoured if u give me suggestion

  15. Kevin Riou Avatar

    Hi , did you have in project to work on opencv with processing for hand tracking or more ?sorry for my bad english 🙁

  16. CitizeenSniips Avatar

    Hey Dan,
    I'm new to kinect/processing.
    is there a way to declare a color value similar to video.get(int(x),int(y)); mapped to the rgb camera?

  17. John_kns Georgiou Avatar

    hello! im trying to use this for a project and i really need to see if there is more of the code. Where did you write the code? when i open the kinect in processing there is already a written code so i need to paste your code in the void draw section with the PImage above and it will work?

  18. Kilgore Trout Avatar

    Hello, Daniel! Great tutorial. I was wondering if it's possible to record/capture the raw data from the kinect and use it as input later on when I don't have access to the camera.

  19. RichardCorral Avatar

    Is there more? I hope there's more

  20. Emilio Vacca Avatar

    This channel is wonderful…simply wonderful…thank you so much!

  21. outlinegorilla design Avatar

    Amazing tutorials. Hope you can make with kinect v2 in p5.js soon.Thank you for all the knowledge share.

  22. Furat Iraq Avatar

    Hey Daniel. I just want you to check this code. I want some feedback and I want you to fix the grid problem for me. Thanks. Reply to me through my Gmail and here is the link. https://www.dropbox.com/sh/ib2450ctb45c9z3/AABICpO1Tpq0xhKiC1Ths8kCa?dl=0

  23. kunjung sherpa Avatar

    Dear Daniel, Could you please do a video on QR code ?

  24. nan zhao Avatar

    Hi, Daniel!Thanks for such a wonderful video.
    I am currently doing projects on LeapMotion,and it is an amazing stuff to work on. Will you do a similar introduction to LeapMotion?

Leave a Reply

Your email address will not be published. Required fields are marked *