PPP-like OpenCV powered DDR controller?

Hi Stepmania developers,

I am a fan of DDR and Stepmania and I've created an OpenCV powered controller for Stepmania.

Idea:
- web cam captures video frame of user.
- based on the user's placement of hands over on-screen regions, arrow keys are sent to Stepmania.

Results:
- This has proven to be very good arm exercise ;D
- Image processing algorithms may need work, but can exceed max combo of 100.

A proof of concept is show below:
- in screenshot
https://onedrive.live.com/redir?resid=B6DF9BA0A3DE9A30%21778
- in desktop screen capture (no audio)
https://onedrive.live.com/redir?resid=B6DF9BA0A3DE9A30%21777

Since PPP never made it to North America, it's hard to find PPP controllers for Stepmania.
This web-cam implementation is a PPP-alternative with standard DDR arrows.

Technical details:
- Proof of concept is implemented using Java, OpenCV (for Java). Proof of concept uses Java's Robot class passes KeyPress and KeyRelease events into StepMania.
- OpenCV can easily use C++

Possible concerns:
- video capture and image processing may take up 300MB-1GB of additional RAM to end-user.
- OpenCV binaries are huge.
- privacy (since web cam is used)

I was wondering if such a software-based controller is a suitable feature for StepMania.
Does it fit in StepMania's concept?

To StepMania developers, please let me know what are the next steps to incorporating such a feature.
I am a developer and could personally port my software to C++.

Thanks,
Garrooo.

It looks like the response rate (not exactly sure how to phrase that) is very bad: you weren't able to hit those three arrows in a row, so I'm not sure if this would work for anything much harder than a DDR 3.

My understanding is that if you write a feature, you can use a GitHub pull request to add it to StepMania, but there's no guarantee it will actually enter mainline.

Thanks for the honest reply tertu .

Yes. Response rate is hard to master. In order to hit the same repeated arrow, the user needs to move out of the region of the screen and back in fairly quickly. With large rectangles, it becomes is a large wavy motion.

I guess this is why DDR hands and feet mode uses 2 tiny little circles instead of 4 big regions.

My bounded shapes are contained in an XML file. Perhaps I will begin simply with 2 circles defined. It would be trivial to feed an XML file with different shapes/keys depending on how smoothly it goes.

In most DDR simfiles, # of hands notes =0. Does StepMania already support hands and feet with camera input if simfile exists? Am I reinventing the wheel?