SDK Announcements

[MrLightTouch]

Now that the MyGesture Editor Beta is stabilizing, our development team has started work on an SDK that will eventually let game, music, and CAD application developers get direct access to some of the surface tracking data thru their OS's USB HID (Human Interface Device) APIs.

The current plan is to provide two classes of data in parallel real-time streams:

Absolute Finger Paths (each finger):

absolute finger position (X&Y)
instantaneous velocity (X&Y)
proximity (total contact area)
contact orientation
contact eccentricity
estimated finger identity (e.g. thumb, index, middle, ring, pinky, palm)

Think of this as 'finger-painting' information.


Extracted Hand Motion (each hand):

finger combination (chord)
hand translation (X&Y)
hand scaling
hand rotation
hand pressure
etc.

Think of this as an 'enhanced' mouse event packet, with a double-bonus of a separate manipulation channel for each hand!

We're actually starting with the Mac OS X implementation because, as noted elsewhere, driver develoment on Mac OS X is about 5 times less arcane than on Windows or Linux! We'ld appreciate pointers from anyone who knows anything about the state of the HID parser API on Linux.

We hope to have an alpha/beta SDK available on selected platforms sometime in August.

If this excites you at all, we'ld like to hear what you all would like to do with such an SDK, to help us focus the development effort.

[John Meacham]

yay! this makes me happy.

[-jeffB]

If this excites you at all, we'ld like to hear what you all would like to do with such an SDK, to help us focus the development effort.

This excites me greatly. :-)

I may or may not be in a position to do serious development with it, but I'd certainly like to experiment with it, especially if it's got a good Java API. (I'm not sure if Java on OS X would be up to it performance-wise, but if I have to drop into C/C++/Objective-C to use it, I'm less likely to do so.)

I'd be very curious about the bandwidth on these channels. Are we talking tens or hundreds of events per second?

One application I'd love to see NOW is a training tool for keyboard users. It would display an image of the keyboard on the screen, and show ovals representing its notion of where your fingers are. For extra credit, since the info is apparently available, it could label each oval with its idea of which finger it represents, and even annotate it with direction and velocity info. I think this would be a big help for new users, and it sounds like you have the data available to pull it off!

[John Meacham]

I personally would love to work on an XFree86 Driver (and perhaps a generalized input extension) which makes this extra information available to X clients if they want it. I think this would open up a huge number of possibilities, such as app specific gestures and most importantly, it will work remotely. Although, this would be easiest if the USB protocol were published or the linux SDK is open source, as linking with a proprietary SDK is not really acceptable for getting integrated with a large open source project. (xfree86).

[John Meacham]

ooh. after the SDK becomes available. I want to write a 'simon' like game. where a sequence of hand motions is shown on the screen and you have to replicate them. it would be a very cool learning aid. I learned to type dvorak by playing the old unix classic 'letters' (also, a wonderful way to learn typing on the touchstream). a similar game tailored specifically for the touchstream would be groovy.

[JerryKnight]

I wouldn't know where to begin, but I find the LP particularly good at photo editing. If the idea of pressure could be transmitted to gimp/photoshop, that would be extremely useful.. Also a "stylus" mode would be great, where cursor position would be absolute and one-finger pointing would be used. The mode would have to be fast swappable (ie. palm slides) for typing. I bet the source to the Wacom linux driver could be canibalized to interface with the LP and send Wacom-like events, making gimp think that it is just a modified wacom tablet talking. But the possibilities are endless in this area. Different tools for different fingers, if the identification is somewhat accurate. You could spread out the effect of a tool by how wide your pointing fingers are spread. narrow - use one finger, wider use 2 or 3 close together. Et cetera.. Wow. Enthusiasm. =)

[JmF]

Absolute Finger Paths (each finger):


* absolute finger position (X&Y)
* instantaneous velocity (X&Y)
* proximity (total contact area)
* contact orientation
* contact eccentricity
* estimated finger identity (e.g. thumb, index, middle, ring, pinky, palm)


Think of this as 'finger-painting' information.


Extracted Hand Motion (each hand):


* finger combination (chord)
* hand translation (X&Y)
* hand scaling
* hand rotation
* hand pressure
* etc.





Maybe my question seems ridiculous, but what about FINGER pressure??
Besides, is there a relationship between pressure and total contact area?:confused:

[fingerworks]

Our sensors, like other capacitive or electrostatic sensors, cannot measure pressure directly. Over a limited range you can infer pressure from total contact area, since the contact area generally increases as additional pressure flattens the fingertip pulp. But the contact area saturates once the finger pulp fully flattens against the bone, preventing discernment of higher pressures.

[JmF]

First, thank you for your prompt reply. :)

Now, i understand the pressure problem.

But, will it be possibile, thanks to the SDK , to treat 'pressure' and 'contact area' (as far as fingers'pulps, their dimension and their various areas can be recognized) as the same?

i mean, there should be a possibility between zero-force and higher pressures to distinguish a kind of pressure thanks to contact area. If the SDK can provide it, this would give us 'only' one more tool, but what a tool!!

[abernstein]

1. how can i get a hold of the current SDK?
2. can the CURRENT SDK be used to program the iGesturePad to act as a graphics tablet, i.e. to input ABSOLUTE XY location from the finger as if it were a stylus?
3. ditto the NEW SDK?
4. what is the DPI resolution of the of the iGesturePad?

[gerb]

Originally posted by -jeffB

One application I'd love to see NOW is a training tool for keyboard users. It would display an image of the keyboard on the screen, and show ovals representing its notion of where your fingers are.

Great idea! Another cool application would be an interactive gesture tutor, for new users of iGesture or TouchStream devices. This program would walk the user through the standard gestures and help them to learn to "speak" clearly to the device. It could ask them to perform a complex gesture (e.g. thumb plus three fingers spread, expand) and show them on-screen how close they came to the "ideal" motion. Of course it would have to bypass the "normal" operating mode and mask the commands produced by a gesture.

Or maybe Fingerworks should develop this... :)

- gerb

[PNit]

I like the idea of the gesture tutor. It'd also be great to get more feedback while you type. eg. Having a keypress sound that alters depending on how close you were to the center of the key. (I think that was mentioned on another thread)

One application I'd like to see is gesture based password entry. You could, for example, hold down a 2 handed chord for 1 or 2 seconds to switch mode. Then you would perform a special gesture with one or both hands (something really complicated that was recorded earlier). If you entered it properly the system will type out the password. Maybe some biometric measurements could also be made for more security (size of the hand).

I can't wait to start playing with the SDK. Is there an ETA for it yet?

[Scoville]

I'm a dot net developer (primarily). Gestures would be one of those things I'd love to be able to use in my applications. Of course, I'm going to be the only one with a TouchStream so it's probably not a big thing for my company, but I write a lot of stuff for home usage and could do some damage there.

[BusError]

Just another "me too". The availability of the SDK is one of the reasons that convinced me to get a keyboard.
I see many fun applications to this!

[amp]

Originally posted by John Meacham
I personally would love to work on an XFree86 Driver (and perhaps a generalized input extension) which makes this extra information available to X clients if they want it. I think this would open up a huge number of possibilities, such as app specific gestures and most importantly, it will work remotely. Although, this would be easiest if the USB protocol were published or the linux SDK is open source, as linking with a proprietary SDK is not really acceptable for getting integrated with a large open source project. (xfree86).

I second this. A closed-source library could become a major problem. Also if fingerworks were to open their protocol specs and/or library I for one would do my best to get a fingerworks surface and help the developement and I think others would probably do the same.

All in all, open-sourcing the linux developement process would make fingerworks much more popular in the linux community than just releasing a closed-source driver would.

-Arthur Peters

[fingerworks]

We will be releasing the library source, but have not yet chosen a license. An open-source license is not out of the question. We are studying the LGPL, Mozilla Public License, and the Apple and IBM open source licenses that address patents as well as copyrights.

What might be nice from our point of view is a license that allows people to modify the source so long as the library is only used with FingerWorks hardware :) Not that we have any direct competitors at the moment. Any thoughts? Would this no longer be considered open source? Do other hardware/driver manufacturers do this?

[BusError]

IMO the sources aren't stricly necessary if the API is good. I don't like the "open source for no apparent reason" and you should definitly make sure that at least your protocols etc are protected.

I don't mind some healthy competition, but I also strongly support protecting the innovators.

[serge.leblanc]

You can choose one of these open licences and add a restriction of use to your materials :
http://www.opensource.org/licenses/

[serge.leblanc]

Personally I will choose the LGPL with closed restrictive with the use of your materials.

[Smeggy]

this is all excellent news, except I'll have to find someone to write stuff for me as I can't program at all :rolleyes:

[luke]

I saw this item on ThinkGeek and decided to search around a bit to see how well it is actually supported under Linux. Glad to see that there'll be an SDK available soon.

Its *very* important that the driver is open source, by which I mean it follows the "Open Source Definition" (http://www.opensource.org/docs/definition_plain.html),

The driver cannot be widely distributed if it is non-free. I would want XFree86 to support it out of the box, without any driver installation. The "only used for this product" idea seems to break the idea of open source, since its hardly fair for a future Linux developer to have to write another driver for a similar product from scratch instead of simply modifiying this one.

As far as competing products, the best defense against competition is just to keep making better products. Any vendor who has to resort to imitation will always be a step behind you.

I'd like to use the iGesture in The GIMP (as another poster mentioned) as well as Blender (a 3D modelling program, blender3d.org).

Any plans for an open-source, maybe non-java, version of the configuration utility?

[JerryKnight]

I disagree. You say non-free drivers cannot be widely distributed, but what about nVidia and ATI? Those linux drivers are everywhere. I personally think that any SDK can be used regardless of whether the source is available. Let Fingerworks keep their source for the time being. All we really want in the short term is functionality.

"The "only used for this product" idea seems to break the idea of open source ..."

It doesn't matter - there is a built in mechanism to restrict use of the SDK/drivers: it would be useless for any other device - similar to nVidia drivers or any other device drivers. And no developer will be making a driver/SDK from scratch for a similar product any time soon. They simply don't exist, nor will they for a long time. Besides, even if they will exist, Fingerworks has no obligation at all to aid developers in writing similar drivers.

I think all too often, we Linux zealots cry for "open source" things because we want to keep evil corporations from hoarding the resources necessary for progress. We demand that companies release their software to us such that we can do practically anything we want to it. Normally, I agree with this. But right now, development by Fingerworks is the best way for this technology to advance. We the people would likely not make any improvements to the actual Touchstream software for a while. We may integrate with systems that Fingerworks has not considered, though. That is what the SDK is for.

"I would want XFree86 to support it out of the box, without any driver installation."

Good luck here. Standard HID drivers would never handle the level of interface we are envisioning for XF86. I would like wacom-like functions, and wacoms need a hefty amount of config mucking for them to work close to right in XF86. And remember, Linux is just one of three major OSes that Fingerworks is working on. Linux support won't even be with the first release of the SDK. And I would guess they are releasing an SDK so that they won't have to write X11 drivers - let someone else handle that.

So for now, we don't need an open source SDK. Nor do we need elaborate licencing arrangements, in my opinion. It would be a nice thing to have when the time comes, but I see it as a secondary concern to the actual functionality of the SDK.

You can use the iGesture with Gimp and Blender, to the extent that the HID mouse/keyboard drivers allow. If you want more, there simply must be custom drivers. Most of what you would need can be done through the gesture editor, though.

I agree that the config utilities should eventually be ported to non-Java, but they work - on at least 3 independent platforms no less. This is remarkable, and platform-specific utilities should only come when there is time and a big need for better performance.

[Edit: fixed spelling]

[serge.leblanc]

"The "only used for this product" idea seems to break the idea of open source ..."

They is true, clause 8 is not respected more.

Most significant is to have the access sources and to have the possibility of integrating our contributions into it.
LGPL licence with a restriction of use seems a good compromise.

In this case FingerWorks and the community profits from the work of each one for the improvement of the SDK.


"So for now, we don't need an open source SDK. Nor do we need elaborate licencing arrangements, in my opinion. It would be a nice thing to have when the time comes, but I see it as a secondary concern to the actual functionality of the SDK."

I'm disagree with this, sources accesses are the most significant thing if Fingerworks wishes that one contributes and thus to support the emergence of the material product iGesture. Personally I don't wish any more waste my time in the development of way for avoided bugs in interfaces. Whereas this can be easily corrected with the access to the sources.

I would contribute only if I have the sources in an opened license, and I accept and understands that FingerWorks would not like to see are work used by one direct competitor.

[luke]

I disagree. You say non-free drivers cannot be widely distributed, but what about nVidia and ATI? Those linux drivers are everywhere. I personally think that any SDK can be used regardless of whether the source is available. Let Fingerworks keep their source for the time being. All we really want in the short term is functionality.
Ever tried using these these cards in a free distribution such as debian (http://www.debian.org) ? Debian runs on 11 different architectures and automatically updates itself over the Internet through mirrors. Neither of these go well together with closed-source products. As you mentioned elsewhere, I can't expect FingerWorks to support all combinations of hardware, operatings systems, and libraries. But given the freedom the open source community does it automatically.

Good luck here. Standard HID drivers would never handle the level of interface we are envisioning for XF86.
Sorry. Let me clarify. By "out of the box" I meant that when I say install X, the FingerWorks pad will work without me having to go to FW's website, click on a EULA, download some file, and stick it in a part of the filesystem that I like to let the system manage. I could imagine maybe discover seeing the hardware, and requesting another package which has more FW software. This package would edit the config files as needed, like most Debian packages (http://packages.debian.org/unstable/) do. Yes, somebody would need to configure it. But if its free software, that somebody doesn't need to be me.

We demand that companies release their software to us such that we can do practically anything we want to it. Normally, I agree with this. But right now, development by Fingerworks is the best way for this technology to advance.
And how does making an open source driver hurt Fingerworks? Do less people buy mice or printers because their supported by an open source driver? Are you saying that this lowers the barrier of entry for future competition? Not really. Here's why: even if the other company adapted FW's linux driver to work with company B's products, they still need to produce a MSWindows driver. Since that would probably be closed source, they could not use FW's open source code, and would have to write their own anyway.

[JerryKnight]

"Ever tried using these these cards in a free distribution such as debian ? "

Umm, I know firsthand that the nVidia drivers are distributed in a bin file that is run with no consideration made of the distribution. If they don't have the driver for a certain architecture, the card probably won't work anyways.

The more I read, the more I get the impression that you don't understand how the TouchStream keyboards work currently. Right now, everything is done through the standard USB HID drivers (keyboard, mouse, et cetera), which are present on every major OS. The keyboards work "out of the box," but if you are talking about things done with the hypothetical evil closed source SDK, then those things will work on the other major OSes because the SDK will be released on the same platforms that are supported now - those with "modern" USB systems. There would be no point in developing for other architectures/OSes because they would not support the required USB drivers, so they couldn't use the keyboard at all. And the only architecture that I see a future need for development would be ARM (pda support someday).

I am very pro-OSS, when it serves a purpose and offers some inevitable benefit (99.9% of the time), but in this case, no real benefit would come from open source because FW is so dedicated to developing for the Linux environment. I am not saying that the SDK should be closed source, just that there is nothing sacred about open source in this instance. An open source SDK doesn't hurt FW at all, but like BusError said, [proprietary] software shouldn't be made open source for no apparent reason.

"... Here's why: even if the other company adapted FW's linux driver to work with company B's products, they still need to produce a MSWindows driver. Since that would probably be closed source, they could not use FW's open source code, and would have to write their own anyway."

Again, I call your argument irrevelant. There are no company B's, nor will there be any time soon. Also, why use the Linux SDK to make a windows "driver"? Why not use the windows SDK? My take on the "only use with FW hardware" constraint is that they don't want to be even indirectly liable for their software on other hardware, if that other hardware someday exists. I don't think it has much to do with future competition, since an SDK is hardly enough to replicate the hardware product or its functionality. The first thing that would be done is a reverse engineering of the keyboard hardware to make an imitation, and like you aptly put it, imitation will always be a step behind, as long as FW actively develops new products. To be blunt, competition will not be an issue for the foreseeable future.

As long as the SDKs are clearly documented and comprehensive (it seems that way from their descriptions), a closed source SDK would be just as useful to developers as an open one. I would claim that any outcry about FW releasing a binary-only SDK would be baseless whining. If you truly have worthwhile improvements to make on the SDK itself, send the ideas to FW stapled to your resume. ;)

(Again, I am not calling for a closed source SDK - I am arguing against the necessity for an open source SDK.)

[luke]

Yes, nVidia does make binary files for any distribution. However, they require manual installation (in this case, manual = apt won't do it for me). On a debian system, there are parts of the filesystem that Joe User should not mess with. Yes, its possible to put a hold an a file, or make my own debian package and install it, or make my own installation of X, but regardless, this is more work than I would need to do with open source drivers.

There would be no point in developing for other architectures/OSes because they would not support the required USB drivers, so they couldn't use the keyboard at all. And the only architecture that I see a future need for development would be ARM (pda support someday).
This is changing all the time. If somebody adopts Linux to run on their toaster, and wants to use to use FW hardware to control poping up the toast, why should FW care? I'm not expecting FW to help them do it, all I'm saying is that when something is open source, unforseen uses are possible.

Again, I call your argument irrevelant. There are no company B's, nor will there be any time soon.
OK.. I think I understand now. What I took away from Fingerwork's post,
What might be nice from our point of view is a license that allows people to modify the source so long as the library is only used with FingerWorks hardware Not that we have any direct competitors at the moment. Any thoughts? Would this no longer be considered open source? Do other hardware/driver manufacturers do this?
was that they were concerned with loosing part of their competitive edge to a future competitor who would not need to spend as much on R&D. This is the argument I have heard for closed-source graphics card drivers: that some company will make a clone of their card. This I never really understood, since as you pointed out, the amount of information in a driver is not enough to recreate the product, merely to interface with it. Point dropped.

As far as liability is concerned: Section 12 of the GNU GPL:
"12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES."

In short, I see no disadvantages to an open source SDK, and several small advantages.

[JerryKnight]

A few closing points to wrap up this rather engaging discussion.

There is no correlation between drivers being source or binary and the ease of installation. Many times, a miraculous open source package is smitten by dependency hell, just like many times, binary packages are incompatible.

The "world is changing" argument is weak at best. I am discussing the foreseeable future, not the utopian era of linux toasters. For now, the list of OSes usable with FW keyboards is pretty much fixed. And in this respect, these "unforeseen uses" are just as possible with closed source as with open. The whole point of the SDK by definition is to make unforeseen uses possible, regardless of whether the source to the SDK itself is available. After all, since the SDK will be so tightly coupled to the operation of the hardware, just about any major change made to the SDK would require hardware to be changed as well. So the only benefit to open sourcing the SDK would be the ability to do very minor tweaking. Is that enough to justify open source? That question is left up to FW, who is probably the most qualified to do the tweaking anyways.

In short, the advantages of open source are arguably limited in this instance, and just because it can be open source with no consequences, this does not mean that it must be open source. Personally, I would not even consider using any SDK package other than the one FW distributes, so for me and everyone like me, open source would be unnecessary.

[fingerworks]

We appreciate all the points brought up in this rather *energetic* licensing discussion. It looks like the safest approach for us to start with is to split the SDK into two tiers:

Hand Motion Tier
The hand motion tier will include the extracted hand motion data, which allows people to easily write zoom/pan/rotation and two-handed manipulation into their 3D CAD, imaging, or many other apps. Since this data stream is really just like an enhanced mouse packet, very similar to other 6DOF motion controllers like the SpaceBall, there's no reason to be restrictive about the interface/drivers. This hand motion data will be easily useful to many application developers, so we will open source the drivers and provide the firmware binaries for free, packaged with the MultiTouch Utilities and MyGesture Editor.


Finger Tracking and Proximity Images
The second SDK tier will be raw finger tracks and possibly raw proximity image data. We expect use of this data to be much more specialized in the near term, e.g. certain artists, musicians and academic researchers experimenting with completely different ways to use the surfaces. This level of the SDK also requires a lot more supporting code, both from us and the developer interpreting the finger tracks. We will use a more restrictive license for this level of the SDK, and may charge a couple hundred dollars for it. If someone comes up with a compelling use of the finger tracks in an open-source app., we will consider a less restrictive license or fee waiver at that time.

[BusError]

/me claps

thats a fair deal for everyone!

Now, when can we get our greedy hands on it ? :->

[-jeffB]

Originally posted by fingerworks

The second SDK tier will be raw finger tracks and possibly raw proximity image data. We expect use of this data to be much more specialized in the near term, e.g. certain artists, musicians and academic researchers experimenting with completely different ways to use the surfaces. This level of the SDK also requires a lot more supporting code, both from us and the developer interpreting the finger tracks. We will use a more restrictive license for this level of the SDK, and may charge a couple hundred dollars for it. If someone comes up with a compelling use of the finger tracks in an open-source app., we will consider a less restrictive license or fee waiver at that time.

This seems reasonable, but "a couple hundred dollars" will put it out of the reach of a lot of artists and musicians, especially amateurs. (And I'm guessing that amateurs will be the largest initial constituency, since not many professional artists have strong low-level coding skills!)

If you're just talking about charging for the source code to this SDK, this objection probably doesn't apply. But if you're thinking of charging for even binary distributions, I hope you'll reconsider.

I have a few ideas of things I'd like to try with the full-bore track/proximity interface, but my free time for development is pretty spotty, and a significant fee would be enough of a barrier to keep me from even trying. Anyone else? Am I missing the point?

[Smeggy]

Maybe a two tiered payment level for personal and professional developer usage, this would open it up to the likes of us minions at a reasonable personal cost while bigger companies/research institutes etc. can pay full price, given they will generally have access to much larger funds than we.

I think this would give everyone a fair shot at getting some really spectacular results from the SDK... the more working on it the more chance of a major breakthrough, and fingerworks will collect more cash from the larger user base and get much more publicity once people find out just how useful these pads are.

Just a thought to appease the restless. :)

[MrLightTouch]

Ya, we can always have the artists and musicians submit an essay about their project in lieu of payment :cool: That's how our UD lab originally got $6000 worth of TI DSP emulator & code generation tools for our first multi-touch prototypes. We just submitted an essay to Texas Instruments and one day all the tools we needed just appeared in the mailbox out of the blue! FingerWorks has now used quite a few of those TI floating point DSPs.

We're behind on the SDK but making progress again. Finally got raw images working thru the USB HID channel today. The sample app. for the Hand Motion tier is gonna be a really cool window move/resize system that beautifully complements Panther's Expose.

[davidstevens]

The iGesture is an ideal gestural controller for use with Max/MSP. (see http://cycling74.com for more info about max, which is now cross platform). I know that there is at least one other Max user who has one of the pads (that's how I found out about it), but I don't think either of us have the programming skills to make the interface object required. So I guess I'll see if I can interest somebody on the Max list in creating an input object. However, I have to say that charging for access to the SDK would probably make this impossible as very few Max users, being artist/programmers, are going to have the funds for something like this just to make one object. So I hope that there will be some way that someone wanting to create a Max input object would be able to get the information they need without having to pay a large fee. (And sorry if I'm misunderstanding the whole licensing thing - it's not really something I know much about)

David

[fingerworks]

The hand motion (free tier of the SDK) should be ideal for things like a Max/MSP interface object.

[Sukandar]

Originally posted by davidstevens
The iGesture is an ideal gestural controller for use with Max/MSP. (see http://cycling74.com for more info about max, which is now cross platform). I know that there is at least one other Max user who has one of the pads

here's another one!
(Hi David, great to see you here. I got me a MacNTouch for my G4 PB)

So I guess I'll see if I can interest somebody on the Max list in creating an input object

I didn't hang around much at the MAX-list lately, but I've been wanting to cook up a Fingerworks-MAX object ever since I heard about the SDK (hoping I can slice out some time from my schedule...)
To me both tiers are interesting, but before getting into the licensing issues I'd wait and see how the first part pans out.

best,
Sukandar

[davidstevens]

yee ha!

hi Sukandar

Have you tried using the pad with max yet? I'm hoping to soon, but my arm has been too bad to do anything very creative for a while.
If you do decide to programme an external, please let me know. I'd be happy to beta it for you!

oh - and i love the mouse driven colour effects on your web pages!

david

[BusError]

So... Are we next week yet ? aren't we ? :D

[JerryKnight]

"I have been wondering for some time now if I could get a raw image from my touchstream for hand-geometry biometric purposes."

It has been pointed out previously that you can get an image of sorts if you slap the pad during the diagnostic sequence. From this you can see that the pad sensor grid is roughly 80x16 (i believe that is for both pads on the LP). Even with crazy interpolation, like what the TS does, the resolution is hardly high enough for biometric use. My hand could look extremely similar to your hand to the keyboard.

[codeczero]

Geez, i figured it was interpolated, but i thought it was more like 160x120. I guess it doesnt really matter then.

[fingerworks]

Beta v0.5 of XWinder and the Hand Motion SDK have been posted:
www.fingerworks.com/download.html (http://www.fingerworks.com/download.html)

We ask for your name and email to participate in the Beta so we can contact you about any important fixes.

Quoting the release notes:

With XWinder, FingerWorks introduces another fundamental advance in human-computer interaction--a simple utility that simultaneously moves and resizes application windows via intuitive motions of BOTH hands. When TouchStream customers drop 3 fingers of both hands on the surface simultaneously, their left hand drags the upper-left window corner while their right hand drags the lower-right window corner. Thus by 'grabbing' opposite window corners with 3 fingers, customers can move and resize applications windows in one intuitive step! Once again, the power of multi-finger gestures eliminates several separate title-bar-click, title-bar-drag, grow-area-point, grow-area-drag steps!

TouchStream and iGesture customers can also move/resize windows with special chords of one hand. With these chords, hand translation moves the window while, hand scaling motions resize the window, and hand rotation motions vary the window's aspect ratio (ratio of width to height).

Though XWinder debuts on Mac OS X only, FingerWorks provides the XWinder source code under the MPL/GPL/LGPL open-source tri-license. This means Linux, Windows, BeOS, BSD, and other operating system enthusiasts can easily port XWinder to their favorite operating system or windowing system.

XWinder also demonstrates the power of FingerWorks' Hand Motion SDK. This SDK will allow developers to design their applications for highly intuitive two-handed manipulations, like panning/zooming a CAD drawing with the left hand while the right hand draws or drags objects over large distances and scales. With the FingerWorks Hand Motion SDK, Mac OS X applications can receive Hand Motion Events, an extended form of mouse event that includes chording, translation/rotation/scaling motion and hand source data. These rich, two-handed manipulations are simply impossible through the standard mouse/keyboard APIs provided by existing operating systems.

[Tigger]

Though XWinder debuts on Mac OS X only, FingerWorks provides the XWinder source code under the MPL/GPL/LGPL open-source tri-license. This means Linux, Windows, BeOS, BSD, and other operating system enthusiasts can easily port XWinder to their favorite operating system or windowing system.
I started to dig in to the XWinder source to determine how much effort it would be to port it to Windows. It looks like it is pretty dependant on Cocoa and Objective C and more of a rewrite than a port is required to move it to another OS. I know nothing about Cocoa, Objective C, or the MAC OS programming environment. If you're looking for volunteers to attempt a port for you, you might want to provide a little more detail about the program's structure and system dependencies.

[MrLightTouch]

No, only the actual window moving/resizing functions are done in Cocoa. These are what's platform dependent. Windows and XWindows may have totally different (C) calls for moving/resizing windows. That's what needs to be looked up and re-implemented, but it could actually be much simpler in XWindows or Windows than the Objective C Accessibility API implementation in XWinderShell.m.

The cross-platform XWinderCalc.c file does all the specific hand motion event interpretation and computes how much the windows should move. It shouldn't have to change hardly at all.

Both Windows and Linux have HID RAW API's with which we will be re-implementing FWHID_HandMotion.c over the next couple weeks.

So what we're looking for help/expertise with is how to grab, move, and resize the window under the pointer in Windows and XWindows.

[fingerworks]

I know some of you have been waiting over a year and probably thought it was vaporware :p, but path tracking capabilities are HERE in the new Hand Tracking SDK v1.5.0! Ya it's weird to start with v1.5.0 but since the utilities and SDK and firmware are all inter-dependent it's simpler to synchronize version numbers.

The Hand Tracking SDK merges the Hand Motion SDK previously released under mozilla/GPL/LGPL open-source tri-license with finger contact/path APIs that provide real-time info on finger location, shape, and estimated identity.

The contact/path portion of the API is provided as pre-compiled static-link libraries, free for personal and experimental use. Source code for the contact/path APIs can be purchased through special arrangement, if needed.

You can view the API documentation here (http://www.fingerworks.com/editutil/sdk/FWHID_HandTracking/ReadMe.html)

The SDK is bundled with the MultiTouch Utilities on the downloads page.

NOTE: Developers who were using the earlier Hand Motion SDK 1.05 or a pre-release version of the Path Tracking SDK will need to study the new API docs closely as many file and function names have changed!

[vectrex]

hmmm
"kFW_TouchStreamTablet_Surface"
"kFW_TouchStreamUltrathin_Surface"

..interesting :)

ps have you considered letting even commerical programs use the sdk? As I fear it could get stuck in a catch 22 of them not being able to justify the cost because not enough people have the hardware, but it's because programs dont support it that people don't have the hardware? Plus the most likely candidate for integration is smaller experimental programs that wouldn't have many funds (ie me :) and I'd assume the core income comes from selling hardware.

[fingerworks]

We're really excited about the new XWinder magnetic edges and tap chords. XWinder is twice as useful now:


XWinder on Windows:_ Tapping right-hand XWinder chord now toggles Windows between Maximize and Restore._ On TouchStreams, tapping the left-hand XWinder chord sends the window under the pointer to the bottom (underneath other windows).

XWinder on Mac OS X:_ Tapping right-hand XWinder chord commands Exposé All (F9), and the left hand chord commands Exposé Desktop (F11).

XWinder on Windows:_ New 'magnetic' window edges help you tile against other windows._ You can adjust magnetic strength in the XWinder Status Menu.

XWinder:_ Improved motion filters help windows move along straight lines.

Tracking SDK on Windows:_ Fixed thread contention stalls in FWHID_enable/disableStreams() functions.

Tracking SDK on Windows:_ No longer needs setupapi.lib or hidclass.lib to link.

[fangs]

Originally posted by -jeffB
One application I'd love to see NOW is a training tool for keyboard users. It would display an image of the keyboard on the screen, and show ovals representing its notion of where your fingers are. For extra credit, since the info is apparently available, it could label each oval with its idea of which finger it represents, and even annotate it with direction and velocity info. I think this would be a big help for new users, and it sounds like you have the data available to pull it off!

I just got a touchstream and found the Mac OSX keyboard viewer palette to be extremely useful in learning the chords. You can turn it on in the International control panel...

[The Juggler]

Hmm, I've been trying to figure this out and can't... could someone who's familiar with what currently works help me out, please?

Nothing complicated... I'd just like to know whether it's possible to capture bitmap proximity data yet, on any platfrom; and what the prospects are for the Linux SDK. (As far as I can tell, there isn't one yet...)

I quite like the idea of turning the TouchStream into an instrument of some kind, and would love to have this kind of data to play with.

[andreas]

I'm interested in the Path/Contact Stream information of my iGesturePad and with help from Fingerworks wrote a Linux device driver to handle this additional stream. It works for the 2.6 kernel.

However I am still lacking the SDK for it, and I wonder what Fingerworks wants to do about it.