Topic awaiting preservation: Image generation using javascript / and a hint for shingebis (Page 1 of 1)

The poster has demanded we remove all his contributions, less he takes legal action.
We have done so.
Now Tyberius Prime expects him to start complaining that we removed his 'free speech' since this message will replace all of his posts, past and future.
Don't follow his example - seek real life help first.

Thanks very much InI - that looks like a promising thing to investigate.

I guess this is a good place to present the results of my investigations into realtime GIF creation, so here goes...

Images in Javascript

As InI explains, images can be generated in Javascript by using an img element where the src attribute is a javascript: url - this Javascript expression should evaluate to a string containing the image data.

This trick is most often used with XBM format images - most notably in Wolfenstein 5K - because the pure ASCII text format is easy to work with in Javascript. Unfortunately XBM only supports black and white images, and its colour cousin, XPM, isn't supported by most (any?) browsers. For that, we need to turn to GIF.

Incidentally, the image has a tendency to get cached against the Javascript URL, which isn't good when we're trying to do animation. As a workaround, I passed the current millisecond time as a parameter to the function.

Compression, and how to not do it

The GIF file specification dictates that the bitmap data must be LZW compressed. This is heavyweight stuff, involving working at the level of individual bits, and we really, really don't want to think about doing that in realtime in Javascript. Unfortunately we can't completely avoid it, because whatever bytes we generate will have to go through the LZW decompressor. We need to engineer our data so that we're not making any effort to actually compress it, but it will still pass through the decompressor intact. Luckily, due to Unisys being arsey about their LZW patent, a lot of clever people have devoted a lot of effort to working out how to do exactly that. (Google for "uncompressed gif".) This meant that I could get away with not properly understanding how it all works - the following paragraphs more or less sum up my entire knowledge.

GIFs can be anything from 1-bit (2 colours) to 8-bit (256 colours). For an n-bit GIF, you start off with a dictionary of bit strings n+1 bits long. The ones beginning with 0 translate to one pixel of the corresponding colour:

00000 = one pixel of colour 0
00001 = one pixel of colour 1
00010 = one pixel of colour 2
...
01111 = one pixel of colour 15

These codes get smushed together in a continuous stream, so you can immediately see that 7-bit GIFs are the way to go - that way we stay on byte boundaries. As soon as we start outputting these codes, the dictionary gets magically populated with new codes beginning with 1 - these represent sequences of two or more pixels. If we were doing the compression properly, we'd drop these codes into our sequence whenever we could - but we aren't, so we'll casually ignore them and stick to the single-pixel codes beginning with 0.

Well, almost. When the dictionary gets completely filled up, we're meant to suddenly increase the length of all the codes by one bit. Disaster! We lose our neat 8-bit boundaries. The remedy for this is code 10000000, which serves as a sort of 'panic button', clearing the dictionary back to its initial state. According to the Awfully Clever Compression Gurus, as long as you press the panic button before you get to (2^n)-2 codes, you'll be alright.

That's pretty much it, except for some niggly implementation details that probably make sense to the gurus, but not me: firstly, that the end of data is marked with code 10000001; and secondly, after generating the complete bit stream according to these rules, it gets chunked into strings of 255 bytes or less, with each chunk being prefixed by the number of bytes in that chunk, and a 0 marking the end of the last chunk.

For my plasma demo, I found that the easiest way to satisfy these rules was with a 7-bit, 258x257 bitmap - I just needed to divide each row into three 86-byte segments, each prefixed by the bytes 87 ("there are 87 bytes coming up") and 127 ("clear the dictionary").

H*w t* survive with*ut zer*

Reckon we're nearly done? Ha. Early on in my experiments, I encountered the bombshell that IE had in store - it refuses to output the byte zero, treating it as an end of string marker instead. (Actually, when I tested it, I'm sure Mozilla suffered from the same effect, but that now appears not to be the case. Maybe I'm barking up the wrong tree here, and this whole section is completely unnecessary.) My eventual target is to get this working on IE, so the question is: can we write out an entire GIF file without ever writing a zero? Yes... mostly. Let's step through the file format.

http://www.dcs.ed.ac.uk/home/mxr/gfx/2d/GIF87a.txt (This site is down at the time of writing, but the Google cache is intact)

• Gif signature: The string "GIF87a". No problem.
• Screen size: Stored as a pair of two-byte words. I chose 515x514, two numbers which don't have any zero bytes in them and are 257 larger than the actual image size, for reasons that will come clear shortly. (Incidentally, PNG stores the dimensions as 4-byte words, so a PNG image with no zeroes would have to be at least 16843009x16843009. That's why I didn't use PNG.)
• Flags: We want to use the global colour map, 7 bit colour, 7 bits per pixel, so the flag byte is 11100110 = 0xE6.
  
  Background colour: What the heck, let's make colour number 1 transparent.
• Spare byte: The official spec says this should be 0, but browsers don't seem to care. The GIMP appears to use it for some sort of aspect ratio setting, and complains about non-square pixels if it isn't 0 or 49. Fine, 49 it is then. (By an amazing coincidence, I know the guy who wrote the GIF decoder for the GIMP. I shall have to ask him about that next time I meet him.)
• Global colour map: Or the palette, as normal people call it. 128 triplets of bytes, and we can keep the range between 1 and 255 and nobody will know the difference.
• Image separator: A comma. GIF files can store multiple images placed at different positions on the screen area, and someone later had the idea of using this for animations. (That person has probably gone into hiding in Nepal now.) We just want one image.
• Image position: We'll put our image at an offset of (257,257) - that's why we made our screen size bigger by that amount. It seems that the final image gets cropped to the portion that's actually used, rather than the full screen size, which saves us a job.
• Image dimensions: 258x257 - the smallest (and therefore fastest) possible image with a width that divides into three equal segments.
• Another spare byte: The official spec misses this out completely, but it's definitely there in every real GIF file I've looked at. Weird. GIMP apparently uses bit 6 as an interlace flag, so we'll steer clear of that and give it a value of 1.
• Flags: I've got this set to 7 in my code, but according to the spec anything with bits 6 and 7 reset should do (use global colour map, no interlace). Maybe I had problems with it set to other values, I can't remember. Sorry.
• Bitmap data: As described above, except we miss out the final zero marker. This is the one place where we have to blatantly break the GIF specification. Fortunately browsers don't seem to mind if we cut the file off here - they've got all the bitmap data by that point, and they're happy to display it.
• GIF End marker: Would be a semicolon, but we've cut it off.

Epilogue

Follow this description, and it will work on Mozilla. But despite all my efforts, I couldn't get it to work on IE. You can generate an actual GIF file according to the above rules, verify that it doesn't contain any zeroes, and display that file correctly in IE - but if you write a Javascript fragment that does nothing but dump that same data out as a string, you get a corrupted image. With this in mind, InI's suggestion certainly falls into place - if I'm getting different results with the same data, it probably means the difference is in the metadata (the bit that says 'this is a GIF image'). I'm not 100% convinced that it's interpreting it as a BMP - it seems to be fetching the GIF palette correctly (although it could just be that the BMP file format stores its palette in the same general area of the file, and the three-byte boundaries happen to coincide) and as far as I remember it correctly determines the height and width from the GIF as well. It's definitely something to investigate, though.

The poster has demanded we remove all his contributions, less he takes legal action.
We have done so.
Now Tyberius Prime expects him to start complaining that we removed his 'free speech' since this message will replace all of his posts, past and future.
Don't follow his example - seek real life help first.

Hello,
Sorry for my recent radio silence. My brother will be dad real soon and I'm with him and his wife during the last days/weeks of pregnancy.

InI: Thank you. I didn't knew the document.open() method had some arguments. It opens a wide range of possibilities indeed.
Btw I think you remember the ugly texture generator I showed you using the data: URI protocol alas only supported by Mozilla AFAIK.
It generated some uncompressed 24bits BMP pictures, the header is bigger than the TGA but not all browsers supports the TGA natively.

Shingebis: generating a GIF image in JS is impressive. Congrats!
And congratulations for the tutorial too.


ps: aaah I miss my Natural Keyboard. The keyboard of my brother is all flat and I type like a pig with it

I don't think we can get away with skipping the zero and ending with the semicolon, because at that point in the stream it's expecting a 'start of next chunk' marker, and so it will read a semicolon (character 59) as 'there now follows a chunk 59 bytes long'. Anyhow, an actual GIF file with those two bytes missing displays correctly - it's only when translated to JS that it breaks.

I've just run another quick test - I took my test GIF file (the one with no zeroes that displays in IE correctly), and renamed it to a BMP to see if the resulting image ends up mangled in the same way as the JS translation. Unfortunately it didn't - it just showed up as a broken image. (I also noticed, when using a hex editor to examine the mangled-JS-version-saved-as-BMP, that the palette takes 4 bytes per colour rather than the GIF's 3 bytes, so they couldn't just coincide by a quirk of fate.) Still, the fact that IE refuses to save the JS version as a GIF must be clue, and I'm very pleased that another pair of eyes has spotted that I'm seriously tempted to try and get my hands on a copy of that leaked IE source code, to see what's going on once and for all...

Poi: Heh, I read your post a bit too quickly just now, and saw "My brother will be dead real soon". You had me very worried for a minute there Good luck to the happy couple!

shingebis: Wow, being dad and being dead are to say the least two really different things.
Have you had a look at Wotsit's Format ?
AFAIR the BMP have a header of 54 bytes, the TGA 18 bytes.

I'll look on my laptop tomorrow. There may be my ugly texture/image generator on it.

Impressive to say the least!
I won't even pretend to try to add anything to the image parts here except one tiny thing on IE.

As far as I know, IE has for a long time has had a wierd way of determine how to treat the filetypes it should display.
It goes something like this (if I can remember it correctly):
First it requests the file(s) and takes the mimetype and some of the actual file-data, from this it decides how to display it according to it's own rules, then, and only then it actually retrieves the file and displays it.

This is why it's so bl*ddy hard to trick IE to open an office-compatible file with anything else than an office application, for instance kick open the download window. IE examines the code, and if it differs from the mimetype or if IE has it's own rules, IE wins.

I have no idea what so ever if this can help, I just thought I'd toss it in the mix
Other than that, amazing work!
/Dan

{cell 260} {Blog}
-{ ?Computer games don?t affect kids; I mean if Pac-Man affected us as kids, we?d all be running around in darkened rooms, munching magic pills and listening to repetitive electronic music.? (Kristian Wilson, Nintendo, Inc, 1989.) }-

I'm a new poster here, but I have some info that might be of help to you.

I've only had the time to give your code a quick glance, but it looks like you use String.fromCharCode for at least some of the bytes that you are generating.

From what I remember (I may be wrong), IE treats unicode strings differently than Mozilla/Netscape does when it encounters them in a situation where it is expecting binary data. Moz will treat String.fromCharCode(125) as a single byte character with the value of 125 when it sees this character in a Javascript url that is in an image src attribute. IE will see it as what it really is... a unicode character, which is actually 4 or 2 bytes, depending on the implementation. So instead of passing the single byte value 125, you're actually passing the values 0,0,0,125, or something similar, depending on endianess and whether it's using 16 or 32 bits.

Again, not sure this is a problem given some of what you said above about IE seeing the image dimensions and such, but just thought I'd put it out there.

-EZ

The poster has demanded we remove all his contributions, less he takes legal action.
We have done so.
Now Tyberius Prime expects him to start complaining that we removed his 'free speech' since this message will replace all of his posts, past and future.
Don't follow his example - seek real life help first.

quote:

---

InI said:

And P01, your mighty javascript texgen was gold, as this one, both do work in Mozilla only in the end.
Except that this one could be made to run on IE, *maybe*.
I now officially give up IE to switch to Firefox: some old habits will
be hard to dump, but the hell to the shittiest browser on earth.
Let it be known, IE belongs to the past.

---

Whoa, awesome InI. I'm glad to hear of your wise decision.

Welcome, EngineerZero! An excellent starter post. Enjoy your stay, give a crack at the 20liner competition. I need more motivation to make a second entry! ;p

Also, I noticed DocOzone is back. That is coolness indeed... has the Doc ever partaken of a 20lines competition? Would he be interested in doing so...?

"Any sufficiently advanced technology is indistinguishable from magic." -- Arthur C. Clarke
"Any sufficiently arcane magic is indistinguishable from technology." -- P. David Lebling

Thanks for the warm welcome.

Wish I had some time to investigate this further right now, but I'm swamped with several other projects. Perhaps this weekend , though... Still, I'm curious to see if shingebis or any one else can make a go of this with IE.

-EZ

This is awsome! It opens a whole new range of possibilites
Is there any chance that this process could be reversed? I mean reading the contents of a bmp/gif/jpg/whatever found in an img tag?

/TwoD

Tada! I've put a hand on the ugly texture generator I did in may this year. That was an über quick test to validate the concept of image generation using the data: URI protocol. It's far from being optimized, but it works. Yesterday I made a 32x32 plasma running at ~13fps using this code as a basis.

Iron Wallaby: Nope, the Doc has not entered a 20liners but he ranked #2 Overall, #2 Function and #4 Aesthetic Appeal at the5k 2002 with OZONE color CUBE.

TwoD: If the images are generated with JavaScript, nothing forbids us to store the RGB values in an array to be altered later and regenerate the picture.

shingebis: I planned to use a similar image generator as a "secret weapon" too.

EngineerZero: Welcome in the Asylum. Enjoy your stay. And don't hesitate to post some tricky scripts. Of course some entries in the December 20lines Javascript Contest - Colors - Entries will be highly appreciated



(Edited by poi on 12-14-2004 23:27)

Latest experiments are here: http://www.west.co.tt/matt/js/giftest/

Inspired by EngineerZero's comments on fromCharCode, I rewrote my brute-force GIF to JS converter to output strings of octal bytes instead - that behaved just the same way though.

Things still to try:

• See if I can break the image before the size descriptor and pallete section by inserting high-ASCII characters there
• Look closely for patterns in the IE output. Inspecting the saved BMP through a hex editor, the end of the file (= the top of the image, because BMPs are upside down) has lots of 0x40s alongside the 0x01s, suggesting that bits are losing their byte alignment.

imagine if this could be taken further to create crude animations. I heard that mozilla team is working on a new tag called the canvas tag, which might be promising in terms of painting to the screen via js.

--- RPG Fan ---

[quickpost]

shingebis: Not all BMP are upside down. IIRC, like for the TGA, there's bit to set/clear to notify if the image is upside down or not.

whisperstorm: If we can build a picture in JS, nothing forbids a freak to build a QuickTime movie in JS too or a SWF or who knows what else.

Actually the canvas tag is an invention of Apple on the Safari browser. It's meant to be used with Dashboard and from what I've heard it should allow to code some complete widgets in HTML+JS+CSS and give the developer the control of the display level of his windows. Well I'm certainly wrong but since I don't have a Mac there's no way I can test it so I didn't read the many papers available on DashBoard and the canvas tag.

Nonetheless I'm rather reluctant to this new tag as it breaks the standards and in essence provides nothing that couldn't be done if little more power/control were given to the developers.

[/quickpost]



(Edited by poi on 12-15-2004 11:38)

poi: I meant grabbing the data in any image, not just those generated by JS. I've tried to find a method to do this but it seems futile...

/TwoD

You guys are freaking insane. All of you. Then again, I guess that's why they call this place the Asylum after all.

I keep meaning to try entering something one of these days, hopefully I'll get around to it eventually.

I was messing around with XBM images a while ago, and I think I was able to successfully get a simple GIF image to work via similar charToString() methods - the downside being it only seemed to work in Mozilla.

However, you may be able to use a document.write() call as someone hinted at with the correct relevant MIME type, and get IE to pick up that it's also an image being dealt with. The future may prove to be interesting.

Have you tried playing with the data: url scheme? you can create any mime type and embed it inlined in a page, I imagine that it's also capable of being created dynamically via javascript as well:

http://www.mozilla.org/quality/networking/docs/aboutdata.html

seems like all you'd have to do is construct the string then set the image url to data: [yourstring]

--- RPG Fan ---

whisperstorm: Thank you for pointing this. Alas, you obviously didn't read my first post, neither watched the URL I posted
Btw, since then I've experimented several things with image generation and have many other ideas to investigate.

[edit] Ian Hixie, developer for Opera Software, has explained that use of the data: URI scheme in late november 2002 [/edit]



(Edited by poi on 12-29-2004 00:08)

Gah forgive my not-reading-ness. I know the folks who started working on the Canvas tag when I was at Netscape... appearantly it's getting revived somewhat - if the code police will let them check in the feature...

--- RPG Fan ---