Drawing a 3D Canvas on top of a 2D Canvas

… is apparently not yet possible on Minefield (as per my conversation on Mozilla IRC).  It’s not possible in other browsers yet either.

<aSydiK`work> does anyone know when trying to change data in an ImageData object, if the data values have to be exactly the same?
<aSydiK`work> I’m reading data that comes out as Uint8Array and need to put it into a Uint8ClampedArray and it’s not going through
<bz> hmm?
<bz> how so?
<aSydiK`work> I’m using canvas.context.createImageData and I’m trying to change the data in there but it’s of type Uint8ClampedArray and I’m trying to throw in a type Uint8Array
<aSydiK`work> I was just wondering if it was even possible
<vlad> aSydiK`work: why would you want do that?
<vlad> the ImageData object requires the semantics of Uint8ClampedArray
<aSydiK`work> well basically, I’m trying to get a webgl-context canvas into a 2d-context canvas
<vlad> to draw from one to the other?
<vlad> in theory drawImage will take any canvas, but that’s broken currently for webgl canvas sources
<aSydiK`work> I’m trying to draw the 3d canvas onto the 2d one
<vlad> I could just fix that for you 🙂
<aSydiK`work> that would be awesome… it’s for processing.js 😀
<aSydiK`work> vlad: what should I do to expedite the process?  did you need me to file a bug or something?
<vlad> there is one already I believe
<aSydiK`work> awesome

After spending the day trying to get a workaround with the image part of the library, I ran into a roadblock.  Unfortunately, this means that the 3D functionality of PGraphics will have to be delayed until canvas.context.drawImage is implemented within the 3d scope.  I’ve been trying to imitate what the already existing 2D PGraphics does in PJS.  2D PGraphics was written to take a new canvas, plaster pixel data on it and then draw the canvas on top of an existing canvas.  I tried doing the same thing through WebGL’s readPixel function.  readPixel returns a Uint8Array data object.  I needed to use an ImageData format… so I built a 2D canvas, called createImageData and tried to set the data.  My final outcome was the following (note – you need a WebGL enabled browser to see):

This is where I was running into formatting issues.  It just wouldn’t render the middle canvas.  So I asked for help, which led me to the #jsapi channel in Mozilla IRC.  And that’s where Vlad told me they were working on it.  Hopefully, it gets in soon so I can work put PGraphics in PJS!

Processing.js and cross browser keycode compatibility

On Friday, I began looking (in more depth) at a ticket that started out as a simple problem of figuring out why an example sketch wasn’t working exactly like it’s Processing counterpart.  It started out as a simple problem but turned into the urge to overhaul our current key input structure to something that works on all browsers.  After much testing and figuring out how each browser treats keyCodes and key strokes, it is my recommendation to add a browser check and account for how each browser handles these items.

I used w3.org’s Keyboard Events and Codes page to figure out what each browser spit out as different keyCodes and charCodes.  This helped me figure out that Opera is pitching out the % keyCode for pressing the left arrow key.  As for key strokes, which was the original problem in the ticket, I used this site.  It showed me that Chrome refires keydown events (on certain keys) when you hold a key down.  This explains why the sketch works properly in Chrome but not in Firefox.  To make it work like p5, we’d have to account for all of that and eventually make everything uniform.

For implementation, I’m looking to port directly into PJS some browser and operating system detection code written in Javascript.  It uses the information from navigator.userAgent.  Strings may not be the best way to deal with them but at the moment it’s the only thing I’ve found that’s simple enough to implement.  If anyone’s got better ideas, feel free to link them to me here.  After I get the browser detection working, I’m moving on to making sure Firefox and Chrome work as p5 does.  Implementing the other browsers may be moved to a later date depending how much I can get done.

My only question to the browser builders out there is… couldn’t you guys get together and standardize all this so users don’t have to worry so much about browser compatibility?  *insert frowning face*

Building mozilla-central/Minefield on 64-bit Windows 7

Let me start off by saying that I didn’t actually start or finish this build.  Maybe I’ll do it sometime on my own time, but during work, it would be too much effort to try to do this build.  I was doing research to try and solve my problem already and I figure I would share what I found with the world.  There’s not much documentation out there for this build so I’m hoping this can eventually help someone else.

To begin, there’s no straight forward linear build for the Windows 7.  Well, the Simple Firefox Build on the Mozilla Developer Centre Wiki provides you with simple enough instructions but the builds for Linux and Mac OSX are way easier.  This is mostly due to Windows not being the ideal development platform.  Anyway following the steps provided in the build instructions, you would end up on the Windows SDK Versions page.  There’s a warning on this page for 64-bit users and it states:

Note that if you’re running a 64 bit version of Windows you may require the “AMD” version of the SDK even if you have an Intel processor, and you may need to deselect the documentation component in the installer in order to avoid an installation error. (This was certainly the case for 64 bit Windows 7 on a late 2008 MacBook Pro.)

That confused me a little, as I couldn’t find the AMD version of the SDK.  I tried installing the regular SDK and it installed without a hitch.  I figured that if there was a OS-bit problem that there would be an error thrown.  Then, the build started and finished.  No problems.  Later, testing the build showed me there were problems.  It didn’t build properly and feedback told me that the build was actually 32-bit.  So, here starts the 64-bit research and the reason I created this blog.

Googling my problem, one of the first links that catch my eye is in the mozilla.dev.build google group.  It’s about build on 64-bit Windows 7 SP2 started by Armen Zambranio Gasparnian, whom I actually recognized was from Seneca’s Open Source Development course (like myself).  First thought was Awesome! Reading on though, I realized he had trouble building it.  Not only that, but his platform is completely different from mine and far more cutting edge.  He’s working with Microsoft Server 2008 along with Visual Studio 2010.  I close the window, for now and move on.

Later, I’m relayed a message originating from Ted Mielczarek, a platform engineer from Mozilla Corp.  The message concerned going back to Armen’s blog for a second look.  I looked through better and found his older blog post.  While the post has mainly to do with the aforementioned platform (MS2008 and VS2010), a comment at the bottom caught my eye concerning WinXP 64-bit building.  The comment mentions involving having to install unofficial patches from this site, which I fail to mention that I’ve seen before through my googling.

Finding this information, I talk to Dave.  Through his info from Ted, along with what I found out, it was decided that the process would take too long and was not really needed (as the 32-bit version of Minefield worked just as well on a 64-bit Windows 7).  So the build was scrapped, but I’ll look into it again if I ever get some free time at work.

Long Journey

We’ve come a long way from when we first started the Processing.js project.  I came to this realization while looking for demos and cool projects to showcase at the OCE Discovery convention happening next week.  At first, I was having trouble finding something new because most of the cool stuff happening with PJS was something we already knew about (for example, Dave’s Audio Build with PJS).  So, trying to find something new was a bit of an arduous task.  When I had given up and looking on to do better things, two different uses/visualizations show up over the weekend.

The first is a visualization of the Evolution of Privacy on Facebook.  This was made by Matt McKeon who is a developer of the Visual Communication Lab at IBM Research’s Center for Social Software.  While the demo isn’t as flashy as some of the stuff that myself and the PJS team have seen before, it’s really impressive as far as seeing the uses for the library.  Speaking for myself, it caught me by surprise.  I started realizing that PJS was becoming more mainstream.

My second example came over the weekend in the form of a game.  Being a gamer myself, I was pretty excited to see Zelda on PJS.  This was created by Derek Doucett at ravenousgames.com.  It’s still a little slow and the PJS code side of things probably needs a lot more optimization but it’s definitely really cool.  Things like this help with determining what we need to work on with optimizing the library code.  It’s really great to see our work being used for amazing stuff like this.

Unfinished Business 0.8 – 0.9 – 1.0

This post is looooonnnnngggg overdue, considering my last post was in March.  I figured I could do this while Minefield was building (but I was wrong considering it only takes like 20 mins for it to build on these machines).

0.8 Release

I really didn’t do all too much for the 0.8 release.  I was busy with other things.  I realize, now, that I should’ve thought more importantly of this project and should’ve made as much time as possible for it.  Unfortunately, there’s not much I can do about the past but learn from it and dedicate my time better.  I did feel badly for not doing as much for the 0.8 release as my colleagues were depending on me and hopefully made up for it somewhat with my work with the 0.9 release.

0.9 Release

For 0.9, I worked on beginShape, vertex and endShape.  I had a barely working product for 3D (untested and unused) before the Processing.js workshop.  It was barely working due to the fact that my changes broke the 2D version.  During the workshop, I was able to run tests and completely revamp the way beginShape/endShape worked.  With the 2D version, the functions were set to draw as the vertices were put in.  I changed it so that it collects that information and then renders at the end.  I, also, spent a whole day on fixing bugs and testing the code.  This was needed due to the fact that it was a large amount of code.  My examples and tests can be found on at the following link:  click here.  (Note: that 3D tests and demos require a pre-release browser that supports WebGL… such as Minefield, Chromium or Webkit).  Here’s a more prominent 3D demo using beginShape/endShape (again you need a 3D capable browser):

After that weekend, I was able to work on converting 2D quad and triangle primitives to work in 3D space.  This wasn’t really too tough as it was merely a wrapper

1.0 and Future Releases

1.0 is a goal for the end of May… however, we will be doing mini-releases along the way to sort out some fixes and release some more needed features.  My first goal is to make that RGB Cube demo working up to speed.  This entails converting a bit of the current endShape code to take in colours for the vertices and render them.  I, also, have to make sure that these objects take normals so they can be lit up in a 3D environment.  After, I plan to work on the PGraphics object and make sure that the fix works with 3D objects being displayed in createGraphics.  My final plan of action before the 1.0 release date is to ensure the PShape datatype is working within the PJS environment.  Of course, I’ll also be fixing any bugs that come along the way.