Skip to content

Updated the complete “Speak with Your Hands” tutorial to use the modern ml5.js HandPose API.Speak with your hands 2.0#1509

Open
colemanliyah wants to merge 13 commits into
processing:mainfrom
colemanliyah:speak-with-your-hands-2.0
Open

Updated the complete “Speak with Your Hands” tutorial to use the modern ml5.js HandPose API.Speak with your hands 2.0#1509
colemanliyah wants to merge 13 commits into
processing:mainfrom
colemanliyah:speak-with-your-hands-2.0

Conversation

@colemanliyah

@colemanliyah colemanliyah commented Jul 11, 2026

Copy link
Copy Markdown

Resolves PR #1483 (#1483)

  • Replaced all older ml5 examples (ml5.handpose(), modelReady(), predict events, annotations) with the new API:

    • await ml5.handPose()
    • detectStart()
    • keypoints
  • Updated every example sketch to work with the current ml5 version:

    • Detecting hand keypoints
    • Printing prediction data
    • Drawing fingertips
    • Adding images to fingertips
    • Moving objects with fingers
    • Controlling multiple objects with multiple fingers
    • Using conditionals with finger positions
    • Creating gesture-based interactions
  • Embedded all example sketches directly into the tutorial using instead of linking out to external sketches.

  • Added updated comments and explanations to match the new code structure.

  • Updated explanations of prediction data to match the new HandPose output format (keypoints, keypoints3D, handedness, confidence).

Comment thread src/content/tutorials/en/speak-with-your-hands.mdx Outdated
Comment thread src/content/tutorials/en/speak-with-your-hands.mdx Outdated

@ksen0 ksen0 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A fe minor comments, but thanks so much for going through and updating everything! @doradocodes if you verify locally as well please feel free to merge.

Comment thread src/content/tutorials/en/speak-with-your-hands.mdx Outdated

@shiffman shiffman left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @colemanliyah thanks for this wonderful work. I made some small comments throughout. Some of my comments apply to the subsequent examples too but I only noted it for the first instance of a change.

Also, the ml5.js style is to use a variable called hands rather than predictions. That might be a little friendlier to the reader, but it's also ok for this tutorial to have its own code style if you prefer to keep it as predictions.

Let me know if you have any questions!

Comment thread src/content/tutorials/en/speak-with-your-hands.mdx Outdated
Comment thread src/content/tutorials/en/speak-with-your-hands.mdx Outdated
Comment thread src/content/tutorials/en/speak-with-your-hands.mdx Outdated
Comment thread src/content/tutorials/en/speak-with-your-hands.mdx Outdated
Comment thread src/content/tutorials/en/speak-with-your-hands.mdx Outdated
Comment thread src/content/tutorials/en/speak-with-your-hands.mdx Outdated
Comment thread src/content/tutorials/en/speak-with-your-hands.mdx Outdated
Comment thread src/content/tutorials/en/speak-with-your-hands.mdx Outdated
Comment thread src/content/tutorials/en/speak-with-your-hands.mdx Outdated
Comment thread src/content/tutorials/en/speak-with-your-hands.mdx Outdated
@ksen0

ksen0 commented Jul 17, 2026

Copy link
Copy Markdown
Member

Hi @colemanliyah I just noticed somethign weird is happening with your commit history (there are many unrelated changes that should not be in this PR) please be sure to rebase/cleanup history, or @ me if you need any help with that!

@colemanliyah
colemanliyah force-pushed the speak-with-your-hands-2.0 branch from 82f3847 to 4414b8b Compare July 17, 2026 16:57
@colemanliyah

Copy link
Copy Markdown
Author

Hi @colemanliyah I just noticed somethign weird is happening with your commit history (there are many unrelated changes that should not be in this PR) please be sure to rebase/cleanup history, or @ me if you need any help with that!

You're so right, sorry about that should be fixed now! Also, addressed your earlier comments in latest commit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants