Refocus – PSIParser

So, I’ve taken a huge step back in terms of blogging.  I just haven’t felt like there’s been much to blog about.  I’m here to refocus because I’ve made what I consider “huge wins” with working on XB-PointStream.

I did feel like I didn’t have much to blog about.  I was having huge problems in the past.  I even approached certain knowledgeable people in the field and they couldn’t help.  It was all a very frustrating experience.  I found the problem and it was one of those really minor ones that can ruin your whole focus.  It was almost as bad as a missing semi-colon… but harder to debug because there were no actual errors being given by the program.  And now I’m sharing so there can be a bit of guidance for those that need it.

First, let me tell you about how I’ve gotten to this point.  I was able to make an example that provided me with the right data by taking binary information and translating it to something comprehensible.  Now, the next step was to use that knowledge and transcribe it into our XB-PointStream system.  This is the point where in I start to have problems.  While plugging in my knowledge and adapting it into parser form, I forget an integral part of reading binary.  Which was:

AJAX.overrideMimeType('text/plain; charset=x-user-defined');

The above code is very important… if missing, the binary values you’re receiving from your XmlHttpRequest will not be correct.  This little bug was my bane for a long while.  I could not understand how I could get it to work with one example and not my new parser.  This lead to my taking apart my parser in numerous points and not being able to see my problem.

After resolving that huge issue, fixing up this parser has been much easier.  I’ve got this example of mickey loading without any normal information.  If you click the picture above, you will see that I have also got it working with our acorn example.  This shows the dynamic properties instilled within the parser.  I’ll have more of our examples up soon.  Soon, lighting will also work as soon as we can adapt our framework to take unsynchronized vertex attributes.  That’s what is coming for our 0.6 Release.


XB PointStream at FSOSS 2010

The Free Software and Open Source Symposium has come and gone (nine days ago) and I’ve still yet to blog about my presentation.  In fact, I just plain haven’t blogged in a while.  I’ve fallen out of habit.  It could be that I’m too busy or that I still don’t like writing about things I haven’t completed.  Either reason is no good and inexcusable.  I’m here to rectify that and talk about my presentation at FSOSS 2010.  I may, also, provide an update (later on in the day) on what’s going on with XBPS.

It was, overall, a good experience.  I’m a little less scared when it comes to public speaking now.  My fellow CDOT workers and I had practice runs.  We also gave constructive feedback on how we can improve our presentations (special thanks to Mike Hoye for sitting through our presentations when he didn’t have to).

The actual presentation ran a lot shorter than I had wanted it to go.  It ended up being only fifteen minutes.  I didn’t want to really get into too much technical detail and was going more for a broad explanation.  The interest came through after when the question section matched the speaking time.  However, looking back, some of those questions should have really been included in the presentation.  By trying to slim down the presentation so much, I made it maybe a little too precise.  In any case, I think it went well and did generate some interest in the project.

My slides are on the web for you to view (they’re made with HTML5): click here.
If you wish to see the slides and hear my talk then follow this link:  click here.  (Also, maybe I should’ve repeated the questions asked.  I’ll go through the question and answer section and put down the questions asked in order on this blog for you to follow along).

New School Year, still working on the same projects

Though do not let that sound resentful; it’s not at all.  I’m still glad to be working at Seneca’s Centre for Development of Open Technology.  It’s great to be able to do work on open source tech (and get paid for it).  Processing.js is coming to a close for several of us.  At least, with the release of 1.0, maintaining the library will become less of a daily thing for most of us.  There’s definitely still bugs to work on and if anyone out there is interested in helping out, just go to this link: here.  Sign up and take a bug to fix!

In the meantime, I’m continuing my work (that I’ve previously stalled) on XB Pointstream.  I’ll initially be working on a reader that converts a proprietary file format to something that’s more usable in the open web (and for our library).  This will probably take me a couple of weeks to get done.  After that, I’ll be working on the many bugs we have filed plus some that are still processing in our heads.  This is definitely going to be a fun semester!

XB PointStream 0.4 Release!

So, I haven’t really talked about my new project a whole lot and now I’m going to change that. This is somewhat a pre-emptive release blog as the release isn’t really finalized and probably won’t be until early next week. I just really didn’t know what to name my blog.  I, also, apologize in advanced as this blog will not be very visual… but feel free to click on the links and I’ve also got a sample demo of the PSI file reader below.

Anyway, XB PointStream is on a good course and we’ve got some functionality working with it so far.  Andor Salga will be implementing items like streaming asc files and webgl arrays for this upcoming release.  As for myself, I’ve been working on PSI files and reading them for the past few releases.  I’ve included the framework for reading PSI files and the XML tags used for said files.  It reads the binary in the file and puts it in a variable for now.  I’ve got a sample demo of it in action (note: due to the size of the file, it does take several seconds… the “test” word will turn into ascii binary).

So I’ve implemented the framework, but the actual conversion of the binary will take a little while longer to implement.  Mainly, it’s due to the fact that I still have to decipher the reader they gave me and make better sense of it to finally convert it.  It’s probably going to take a few more releases before I get a finalized working version.  There’s also still plenty to do in the meantime though.  Like with the current release, I’ve also implemented a few easy functions.  One was a simple default function that sets some of the variables back to their default values.  The other was taking sephr‘s tinylog lite logger from Pjs and put it into XBPS, so that we have a simple logging mechanism.  This will probably be changed later to our own custom one, once we have time to implement it.

I’m going to leave you with a bunch of links relating to the project:

Wiki Page – clickity
Github Repo – clickity
Ticket System – clickity
Twitter – clickity
Blogs – clickity

Toronto WebGL Community

Left to Right: myself, Benoit, Andor and Matt

This is the Toronto WebGL Community!  Or at least the people we know that currently actively work on it…  if there’s more of you out there, feel free to let yourself known and get involved by posting on the comments of this blog.  I’ll make sure the right people see your stuff.

Anyway, this picture was taken at the Mozilla Toronto Offices.  Yesterday, I spent the afternoon with some of my co-workers talking to Benoit Jacob about the WebGL implementation within Minefield.  We were doing some profiling and giving feedback on what changes could be made to provide better support for those using the WebGL API through Firefox/Minefield.  So if any of you have suggestions, specifically more about how it’s implemented in Minefield and less about the actual WebGL standard, feel free to leave a comment below and I’ll do my best to get the idea to them.

I, also, learned a couple of neat tricks.  One such is a built in profiler in Linux that I could use on my desktop at work.  It will make bottlenecks a lot easier to find.  The command is perf record, which actually only came out to Ubuntu on a recent release.

We discussed other performance boosters, like TypedArrays.  This is a new proposal to the Javascript language and will be somewhat monumental considering JS is a type-less language.  This introduction will help improve WebGL code and will pave the way for the final release of WebGL.  It just has to be approved by the standards committee.

***NEW*** Toronto WebGL Mailing List (created by Benoit) – clickity

Manually creating a PNG…

… pixel by pixel.

Like I’ve said before, most of the Processing.js team are shifting their main focus on different projects over the summer.  Some of the members are working on Popcorn.js, a Javascript video player library that uses the HTML5 video tag.  Andor Salga and I are working on imaging 3D point cloud data for Arius3D.  Andor already has a working prototype gallery for what we have to do with our point cloud data; his blog about it is located here.

I have been working on the non-3D spectrum of this work and trying to improve the load times for our prototype.  Well, that’s what I’m going to try and accomplish in the end anyway.  I’ve been looking for ways to solve this problem and the first method I decided to try was server side scripting using PHP.  While PHP isn’t a hard language to comprehend, I soon ran into problems concerning the use of the data I was reading from file.  The only reason I was using PHP was to try and read the file data to give to Javascript.  If trying to extract the data was this much of a hassle, there was no point in using it.

I moved on to just reading the file straight from text using XMLHttpRequest.  I had problems with putting setInterval and setTimeout methods within the code to try and run asynchronously.  The problem was that I didn’t need them!  After I removed them and just called the function on its own, I got the stream and conversion to work… but very slowly.  An example of the stream can be viewed below… be warned, though, it does take a number of seconds.

I was talking with notmasteryet, one of the audio wizards that came up with this hack of streaming byte data into a png for Text to Speech use within browsers, and he told me to use bit shifting instead of string conversions to plug the information into the png.  This would of course make the conversion MUCH faster.  However, something is wrong and not working as it should.  I’m writing this blog not due to accomplishment but more of a bookmark on what I may have to continue with later.  I’m moving on to dissect the PSI Reader given to us by Arius3D and trying to convert it to Javascript to be used for on the fly conversions.