Stoopid Net Trick
Dennis Crowley // dens@dodgeball.com

Interactive Telecommunications Program @ NYU
Networked Objects, week 5 // February 25, 2002


Description
1. Use the Cobox to pull content from a web page (e.g. stock quote, weather) and into the BX24.
2. Represent this data in some visual/audio/physical fashion using the BX.

Ingredients
1 NetMedia BX-24 microprocessor ($49)
1 Latroniz Cobox Micro embedded server ($49)
1 active ethernet connection
10 hours of time to lock yourself in the p.comp lab

How it Works
It's taken me forever to get this far... and while I sit here and try to write it up it all sounds so simple. Most of the problems I ran into had to do with the BX flaking out on me - errors that I was able to fix by removing "debug.print" statements and removing comments I had written (read: stupid things that shouldn't make a difference, but always do when it comes to the BX).

Anyway, here's the Top 5 Things That Drove Me CrazyTM... (open the source code in a new window and follow along for best results)
1. The Cobox has to speak to the web server in order to get things started
Initially, I though that the Cobox would work just like a web browser - I ask for a web page and the server will spit some HTML back at me. With much help from Tom, Shawn and Ahmi, I realized that when a request for a web page is made from a browser to a web server, the browser (a) makes a connection and then (b) specifically requests the page via a GET command. If you look at the source, you can see that the Cobox has to make the connection first (connectString = "C128.122.253.189/80" & chr(10)) and after a connection is made, request the page (GET /~dc788/cobox/cobox.html" & chr(10) & chr(13)). In this case, the connect string is the IP address for stage.itp.nyu.edu, port 80 and the GET request is looking for a specific flat file on my stage account.

(ps: check out the flat file - it's literally just the number "32", no HTML or anything. I figured it was probably best to start with the simplest document possible so I wouldn't be stressed by the parsing of HTML, etc.). Eventually, I'll replace this .html file with a perl script or .asp file... something that is pulling data - whether its the temperature in NYC or the score of the Red Sox game - from a database and presenting it in some kind of dynamic format.)

By the way, the best way to test your GET string syntax is to open a telnet window, connect to stage.itp.nyu.edu (er, "telnet 128.122.253.189 80") and once the connection is open, type the GET string. At least you'll see what you're going to get back when you finally do connect.



2. Get the input and output buffers right.
I'm pretty BX-stoopid when it comes to setting up the size of the buffers for serial in and serial out. When I first started messing around w/ the Cobox, I was setting the buffers at ridiculously high settings (300 bytes in, 100 bytes out, etc). With Daniel's (aka SpiderDan) help, I was able to connect a lot of the mystery BX problems I was having to my carelessly set buffer size. We set the buffers (both in and out) to 60 and methodically worked our way down - 55, 50, 45, 40, 30 - until we were able to get most of the app working again.

This next part may or may not be right, but I figured that each character equals a byte (right?) so I should probably set the buffers only as high as the maximum number of bytes I was expecting to get from the BX via the Cobox (read: what was coming back from the web server) or what the BX was sending to the Cobox (read: the GET string that requests the html page).

I knew that my flat html file would only return two bytes to the BX and thus I set the input buffer to 11 bytes (remember, the BX needs 9 bytes for 'housekeeping' and 9 housekeeping + 2 from the "32" in my html = 11 bytes). As for my outgoing buffer, I counted the number of characters in my GET string (including spaces and special characters). Total number of characters is 31 + 9 for housekeeping = 40 bytes. For some reason, 40 didn't work (BX black magic!) so I bumped it up to 42 and it mysteriously started working again.


3. Convert the incoming bytes into readable characters
After I fixed numbers 1 and 2, things actually started working and I was able to pull the "32" from my flat html file into the BX for the first time (which is much more exciting that lighting your first LED). The main problem I was running into now was that the Cobox was spitting back random bytes - numbers that didn't make any sense to me at all. I tried connecting to a number of different sites (amazon, dodgeball, yahoo, stage) and started writing down the patters of numbers I was seeing - the same numbers were emerging over and over for the same sites.

See, now this sounds so stupid in hindsight, but on Tom's suggestion I ran a simple chr(dataByte) conversion to turn the bytes into readable ASCII characters. Once the data coming back was converted from byte value to ASCII character, I was able to see that the Cobox was spitting back "C128." or "<!DOC" and sometimes even "32" (the magic number!). After digging around, I figured out that the "C128" was the Cobox connection string being spit back at me and the "<!DOC" was the Cobox not being able to locate the file I was requesting w/ the GET command (e.g. Check out this page. Look at the source. Notice the first three characters?)

Anyway, converting the bytes to ASCII characters made the whole thing debuggable.


4. Give the Cobox enough time to connect to the server
When I saw that the Cobox was spitting back the connection string and "file not found" errors, I started to figure out that maybe I wasn't giving the Cobox enough time to connect to the server or receive a page after issuing a GET command. After all, when you telnet to stage it takes a second or two for the client to make a connection to the server. So... I threw in a few instances of "call delay" - one after the connection string is sent through the output queue and one right after I call the GET command. Once these were in place, things started working with relative consistency.



5. // This Space Reserved for the Problems I Run Into Tomorrow //
Okay - it's one thing to get the flat file text into the BX, but I still have two more problems to solve...

(1) How do I make this use of the data? Although my flat file displays a solid "32", the BX is pulling back two bytes, which in ASCII, translate to "3" and "2". I need to find some way to concatenate these bytes and then convert them to an integer so I can control some bells and whistles with the data. The converting is pretty straight-forward, but as of right now, I have no idea how I am going to make "3" and "2" become "32".

(2) I'm still stuck with this flat file. Ideally, the Cobox should be hitting some Perl script or some .ASP file that is either hitting a database and returning a value or stripping data from, say, weather.com and dumping just the temperature data into a flat file. As of right now, every time the Cobox hits my flat file it's going to pull back a "32" since that's all the page knows. 32. 32. 32. 32. 32. Boring. I need to find someway to dynamically update the content on that page.

Source Code
Click here to view the source code

(c) 2002, dennis crowley