Topic: Creating "Buffers" - problems with history and coookies

Hello!

I am trying to extend my sites session handler to be able to track users between different windows. This will let me do custom message and lastpage handling for each open window instead of each user. I am calling the open windows "Buffers" for the sake of simplicity.

I have been working on this for several days, and have been "inches" from a solution several times... Every time, I seem to run into a wall right before success

I have a server side database table of buffers that are refrenced by sessionid and bufferid. It works perfectly. So far, I have everything working but the javascript that starts a new buffer when the user opens a new window. Sounds easy? I'll list all the attempts I have made and the problems I have run into. Mabye someone will be able to "break though" the wall

Server Side:

Match http referrer header:
This method works like a chain. I store the contents of the the users current page in the buffer database. On the next page, I look for rows in the database that match the referrer header. New buffers are created when no rows are found to match (the user "branched").
This method worked perfectly untill the user used their back and forward buttons. Consider this:

1) User visits page X, Y, and then Z.
2) User hits "Back" (on page z)
3) Page Y's referrer is page Z, right? Wrong! the referrer is page X, and the chain is broken.

Client Side:

Split buffer when history.length == 1 (1st try):
In this case, the work of starting a new buffer is done client side when history.length == 1. A new bufferid is created, and assigned to a cookie. The cookie setting function runs in a loop, and sets the buffer coookie every three seconds.
The problems with this are obvious. When multiple windows are open, the buffer coookie keeps getting updated with different bufferids from each window. When the user requests a page, the cookie sent along with the request may be from a different window, and the users buffer gets confused.

Split buffer when history.length == 1 (2nd try):
Same as the preceeding attempt, but with the addition of a mechanism in place to deactivate the loop when the window is in the background.
The problem is that JS (as far as I know) doesn't have a function to return the window's focus state. It has onfucus and onblur event handlers, but how do I tell if the window was loaded in the BG from the start??? Since I don't know the window's intial state, the event handlers seemed to be of little use.

Split buffer when history.length == 1 (3rd try):
Same as the 1st attempt, but instead of a loop to set the coookie, the cookie is set on the onunload event (right before a new page is visited). This should work perfectly, but from all my tests it seems that there is some bug in JS which keeps a cookie set from a function attached to the onunload event from being set along with the http headers till two pages later!
According to my tests, when you set a cookie from inside a function attached to the onunload event handler, the cookie will not be "visible" to the webserver serving the subsequent page!
If anyone knows a way to work around this, please tell me!



As you can see, I am almost there.
If anyone can find a way to get one of these methods to work, it would be wonderful!

-Robur

Could you elaborate on what you are attempting a little more.

You have your session which you are tracking, when the user opens a new window are you saying a new session is spawned and now you are attempting to track this new session?

I am not exactly sure where this buffer variable is needed. If you are tracking the user via a server session you should have access to all of the users local variables no mather if the user spawns a new window and are working between 10 different windows.

What language are you using on the server side and can you give a more specific usecase?

Dan @ Code Town

WarMage:

I am using php on the server side, and it is working perfectly.
Each user is assigned a session id which refrences them to a row in the database. The database contains all the standard user information, as well as lastPage and message.

For an example of what these two rows do go to my classifieds site's submission process:
http://safarri.com/submit
1) Select regular classified (requires registration), and you will be taken to the login / registration system.
2) A friendly message shows at the top of all registration pages, and once you are finished you are redirected back to the submission page no matter how many pages the registration process took. It is like a smart PHP $_SERVER['HTTP_REFERER'] coupled with message handling. :-)

The problem comes when users have multiple windows open. If you are on the browse pages in one window and go through the submission process in another, the lastPage and messages get totally messed up.
What I did was to split those the lastPage and message colums into their own database table, and refrence them to each user AND each open window. (each SESSION can contain as many BUFFERS as the USER has OPEN WINDOWS)

I have everything working except for one (seemingly trivial) point: How do I create a new buffer ID when the user opens a new window? See the above post for my attempts...

-Robur

Have to add one more attempt....

Split buffer when history.length == 1 (4th try):
Same as the 1st attempt, but instead of a loop to set the coookie, I attached a function to the pages onLoad event handler which checks if history.length == 1, and if it does redirects to a page that sets the cookie server side - and then serves a blank JS page that calls history.back();
The problem with this attempt is the redirect. Though I have tried updating history.location, and using document.write to add a META REFRESH tag to the page's header, both methods overwrite the current page in history. On the blank JS page that calls history.back() nothing happens, as there is nothing in the history object!!!

JS drives me crazy

-Robur

It seems like you are talking about continuation management.

At work we use Cocoon which handles the whole state machine aspect of this. Maybe a little discussion on how Cocoon handles their continuations might give you an idea on how you would want to set things up.

Just a little note, in the application we have created there are areas where messages get confused. We are using a mess of nested iframes with areas that spawn new windows and processes that happen in parallel so many systems will have trouble attaching a message where you want it all the time. The trouble is actually the developer not coding things correctly, but just saying that it can be confusing even when using a somewhat developed framework.

With a continuation each request gets its own continuation id, and the continuation manager which controls the creation and destruction of all of the active states.

The easy way to look at this is that each different request creates a different id and a copy of all associated variables, when coming back through on a request (when the user presses submit) the continuation id is also sent and used in combination with all of its associated session values (such as lastmessage).

This is all done in memory without saving to the database. There might (and probably are) disk writes that are happenning but that is all controlled by the serialization mechanism of the server.

Another thing to note is that not all pages use a continuation, only those that will need to keep track of data related to the current request chain.

An overview might look something like -



As you can see each page has a display, which is the message that is currently being shown, then there is the id of the page which is stored on the page (either as part of the form's submit URL or in a hidden field), the current page when submitted will create a message m, which is the message that will be displayed on following pages.

What this gives you is the easy use of back and forward (well at least back as forward in a submission process tends to resubmit the data) which will take you to a page that has an id embedded in the page.

Since all of your local data is associated with an id and you can easily tell which message belongs to who by keeping track of each continuations parent.

What this does (that your method does not) is keep a whole lot of garbage data present, as you will be spawning continuations that will all have to live for the life of the user's session (or you will have to develope a garbage collection scheme).

I know that all of this writing doesn't directly solve your problem in a if you do X you will be set kind of way.

I don't think I could give you a direct solution without seeing your code as just using words relegates this to an exercise in theory.

You are attempting to do something that a lot of bright people are spending their time working on, kudos in that respect.

Dan @ Code Town

(Edited by WarMage on 06-01-2006 03:57)

WarMage:

Thanks for the interesting information. I think my method will need garbage collection too, though not as much.
I added a field to the buffer table called "bufferLast" thats a unix timestamp updated every time the buffer table is read. I was thinking of deleting all buffers older than a day or so. Another thing I expirimented with was keeping all the open bufferids in a comma seperated string in the cookie most recently viewed first. If I can get the base mechanism working, I may try to finish that :-)

I was hoping an "excercise in theory" would be enough, but I guess not. Here is my current JS Code (mostly an updated version of "attempt 3"):
The only problem with this script is that the cookie called "atibuf" is NOT sent to the "link target" page (the page the link clicked points to) as far as I can tell. Once a linked is clicked ON that subsequent page, the cookie is sent as it should be.

<script type="text/javascript"><!--
	var ati_buffer = "'.$_SESSION['bufferId'].'";
	if(history.length == 1 || ! ati_buffer) {
		ati_buffer = "";
		pool = new String("abcdefghijklmnopqrstuvwxyz123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ");
		var i = 0;
		while (i < 15) {
			rand = parseFloat(Math.random()) * parseInt(pool.length);
			ati_buffer += pool.charAt(rand);
			i++;
		}
	}
	function ati_buffer_finish() { //Called OnUnload
		document.cookie = "atibuf=" + ati_buffer + "; path=/; domain=.'.$_SERVER['HTTP_HOST_TOP'].'";
	};
--></script>

Basically, I am using the cookie as a "very short term" variable preservation technique. Is there some other mechanism I could use for this? Can JS modify post data or set http headers on the fly? (Get data is excluded, as it will mess up all my beautiful URLs)

-Robur

EDIT:

Sorry... I just had to update my code...
The new version almost supports keeping a list of the current buffers, but I need a way to detect if the user clicked a link or closed the window. If the window is being closed, I will remove the current bufferId from the list.
It almost seems like I should create two cookies, one with the current buffer that expires instantly and one with all the other buffers that doesn't ecxpire, and then delete all the buffers from the database that don't match either cookie for the current sessionid... will have to play around with that.
It almost seems it's so complicated that i'd be better to just delete the old buffers - after all, I will have to do that anyway because lots of my visitors are using windows so their browsers are sure to crash sometimes (just kidding)

<script type="text/javascript"><!--
	
	//Init Vars
	
	var ati_buffer = "";
	var bufferString = ati_getCookie("atibuf");
	var bufferCookie = "";
	if(history.length == 1) {
		pool = new String("abcdefghijklmnopqrstuvwxyz123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ");
		var i = 0;
		while (i < 15) {
			rand = parseFloat(Math.random()) * parseInt(pool.length);
			ati_buffer += pool.charAt(rand);
			i++;
		}
	}
	else ati_buffer = "'.$_SESSION['bufferId'].'";
	
	//Set cookie
	
	function ati_buffer_finish() { //Called OnUnload
		if(bufferString)
			var bufferSplit = bufferString.split(",");
			for (i=0; i<bufferSplit.length; i++) {
				if(bufferSplit[i] != ati_buffer) bufferCookie += "," + bufferSplit[i];
			}
		}
		document.cookie = "atibuf=" + ati_buffer + bufferCookie + "; path=/; domain=.'.$_SERVER['HTTP_HOST_TOP'].'";
	};
	
	//Helper Functions
	
	function ati_getCookie(name) {
		var nameEQ = name + "=";
		var ca = document.cookie.split(";");
		for(var i=0;i < ca.length;i++)
		{
			var c = ca[i];
			while (c.charAt(0)==\' \') c = c.substring(1,c.length);
			if (c.indexOf(nameEQ) == 0) return c.substring(nameEQ.length,c.length);
		}
		return false;
	}
--></script>

(Edited by robur on 06-01-2006 05:01)

(Edited by robur on 06-01-2006 05:06)

I really can't say that I like having JS handle your session management.

Were I to do this I might create two classes for the continuations

class Continuation {
  var $id;
  var $parent_id;
  var $message;

  ... Getters Setters and Helper Functions Here ...
}

class ContinuationManager {
  var $continuations = array();

  ... Getters Setters and Helper Function Here ...
}

Then I would pop the manager into a session (if it is not there and all that jazz).

$_SESSION['continuations'] = &New ContinuationManager;

Then on each new page you would need to generate a new continuation and link it to its parrent continuation. This could all be done in the continuation manager with some nice helper functions.

$_SESSION['continuations']->generateContinuation()

access would be pretty easy as well.

$_SESSION['continuations']->getLastMessage()

In this way you will have all of your session management in your server side code and not have to worry about buggy browser support for JavaScript or even those who do not have JS enabled.

Dan @ Code Town

(Edited by WarMage on 06-01-2006 06:40)

WarMage:

JS isn't handling my session manager, all its doing is assigning a new buffer id whenever the user opens a new window. I already have all my server side code working perfectly.
I don't like having JS do anything, but it seems there is no help for it.

You mention a $id and a $parent_id, then you say:

quote:

---

Then on each new page you would need to generate a new continuation and link it to its parrent continuation. This could all be done in the continuation manager with some nice helper functions.
$_SESSION['continuations']->generateContinuation()

---

How does the generateContinuation() function work? the parent ID needs to be passed to the child page somehow... Hidden form fields? the Referer header? That is what this thread is about :-)

I already tried something completley server side, and nearly got it to work - see "Match http referrer header" in my first post.

Thanks for your seggestions,

-Robur

You are only worried about generating a new id on a new request.

The current ID is generated at the top of the page and then it is added to a hidden form field.

So when you get to a new page you will have the previous pages id in your $_GET or $_POST and you would use this to generate your message and create your new id.

Dan @ Code Town

WarMage:

Form fields are only passed along when their form is submitted. Form fields are not passed along when a link is clicked. I have used JS to detect link clicks, set the link target to the location of a hidden form, and then submit the form, but this seems impractical and would not work on pages with forms on them.
It seems that you are suggesting manually adding a hidden form field to every form on my site, and having the messages lost whenever the user clicks a link. This might work, but it doesn't seem very elegant :-)

Just got a new idea:
onUnload: append an anchor to the page's URL (#bcid=1234567889). I should be able to parse the anchor from the referrer header on the subsequent page. Not sure what will happen when the user hits "Back", though. Will the anchor show up in their URL (If so, this method is disqualified)?
Another disasvantage of this method is that it relies on the referer header, which I have heard that some firewalls, proxies, and security systems "strip out".

-Robur

I got it working!

When you attach your cookie setting function to the onClick event, everything seems to work.
I am currently getting it to work across domains with iFrames, and am working out the bugs in my server side code.

I will post a link to a test site soon!

-Robur

I keep getting more confused about what you are trying to do. Could you create a diagram of a typical use case for a user? I would like to see where these messages are being displayed and what paths the user might take through the application.

Dan @ Code Town

Here is a "typical use case". THere are many more uses than the one here, but it serves as a good example:

So basically you are attempting to get the user back to "Regular Classified" after they have completed the registration process?

Also you set a message at the same time that you set the lastpage. What is an example of the message you would be setting, and why would this be used on the registration page?

Dan @ Code Town

Wait a bit, and I will show you :-)

{This post will be edited once I get my buffers working correctly (hopefully today)}

-Robur

{The forum said my post was too old to edit... :-) }

I got it working!

The example can be seen on my classifieds site:
http://safarri.com/submit
(click "regular classified" to go through the registration process)

This is only one sample use, and I admit that it could be accomplished with hidden form fields. From the programmer's perspective however, my version requires much less code. All I had to do was call a redirect function on the submission page that set the message, and set a deadEndPage = true flag on the registration pages.

So far, I have everything figured out but the multi domain part.
Once I finish that, I will write a tutorial on this and post a link to it here!

-Robur

Could you explain why this wasn't possible just using the session?

You could have your redirect logic first set $_SESSION['lastpage'] = 'thispage' before you perform the redirect. And then when you have completed the registration just redirect based on the stored value.

I must be missing something.

Dan @ Code Town

I used to do it that way, but it completley falls apart when the user is doing multiple things at once with the same session.

Using my method you can be browsing, submitting, registering, and using the help system at the same time with the same session, and still have everything work perfectly! It integrates nicely with the modern concept of tabbed browsing.

Since ATIServer (my PHP powered webserver) serves tons of different domains and supports multidomain "network" logins, storing window related items in the users session db could be a serious problem :-)

-Robur

Gotcha.

Glad you solved this problem.

Dan @ Code Town