Developing 3rd party javascript code/widgets

D Hayes

Anyone have experience in this area?

Along the lines of Google Analytics embedded code, Crazy Egg heat maps etc.

It's something I'm interested in pursuing, but I have a lot of questions:

- what technology stack would you recommend? I come from a Windows stack background, but have been told that the MEAN (mongodb, express, angular, node.js) stack is more suitable for these widgets. Better async, concurrent users etc.
- how to prevent cross site scripting and other security gotchas?
- best way to create the actual widget js code, using async loading etc.

All advice or links to good tutorials appreciated. Bonus points for showing me some good ones in action!

colm_c

Depends what you want the widget to do, for example Google Analytics js isn't that complex, it mostly calls a 1x1px gif from the server which has a cookie, and the server/cookie does most of the work.

There isn't one size fits all to be honest, same goes for the backend/async services. I've seen rails apps for backends, backend free options etc.

As for the JS:

- keep it as small as possible,
- testing on multiple platforms
- use closures
- don't pollute the global namespace
- don't relying on third party plugin versions (jQuery etc.) being available.

Graham

For the client-side, there's a book specifically on the subject that may be a good starting point for you, 'Third-party JavaScript' published by Manning. Ideal for the widget side of your project.

Have a look at socket.io for real-time cross-browser data transfer it looks pretty good. I came across it while I was testing out sails.js, socket.io is a core part of sails.

Sails.js is definitely worth a look for the back-end and API part of your project. It would be the back-end ME?NS part of your project, mongodb, express and node.js with socket.io with the added bonus that you can swap out the mongodb part with a couple of other DB back-ends. You'd probably then add something like Angular for the web-app part of your project.

Firebase may be worth a look as an example of what you can do with real-time web apps but personally I prefer something you can move to another provider if required. Maybe you can now with Firebase, haven't looked at it in a while.

Talisman

Graham is right, the Third-party JavaScript book is a good starting point for you on the client side.

To do it right you're going to need to create a SDK to interact with your server side API just like Google Analytics. Google Analytics uses a cookie to store the clientId, every request to the API uses the clientId to track the client. If the client clicks a link to another domain how does the tracker follow them? What if a user shares a link containing a tracking code with other users? Read through the developer guides to see such potential pitfalls and the solutions Google have implemented.

If you intend to put the system into production you should also consider the following:

In terms of the server, by default Windows server is limited to 3000 concurrent HTTP connections. This limit can be increased by updating a registry key (HKLM\SYSTEM\CurrentControlSet\Services\HTTP\Parameters\MaxConnections), the upper limit on inbound connections is 2,031,616. The outbound port connection limit is 5000 by default, the upper limit is 65,536 and is set by updating another registry key (HKLM\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\MaxUserPort). For Linux it's best to do a search to find what parameters to edit, the location of the files to be edited can vary but generally are /etc/sysctl.conf and /etc/security/limits.conf. The amount of RAM will ultimately determine how many connections the system can handle concurrently so a 64-bit OS would be recommended.

Node runs on a single CPU core so you will need to use the cluster module to spawn workers to handle requests. Alternatively if you don't want the additional level of complexity in your code, you could use Nginx as a load balancer forwarding requests to separate instances of your Node application each of which would be listening on a different port, I have done this in the past and it has worked well.

Another consideration is Garbage Collection, I know that V8 had issues with GC when more than 512MB RAM was in use, the work around was to essentially disable it and implement your own solution. I'm not sure if this has since been improved.

D Hayes

Graham wrote: »

For the client-side, there's a book specifically on the subject that may be a good starting point for you, 'Third-party JavaScript' published by Manning. Ideal for the widget side of your project.

Thanks, will check it out.

Graham wrote: »

Firebase may be worth a look as an example of what you can do with real-time web apps but personally I prefer something you can move to another provider if required. Maybe you can now with Firebase, haven't looked at it in a while.

Cheers - I had actually contacted these guys a few months ago. I'm still trying to figure out where this service fits into the stack though. Any ideas? Does it cut out a lot of the up front development?

Talisman wrote: »

In terms of the server, by default Windows server is limited to 3000 concurrent HTTP connections. This limit can be increased by updating a registry key (HKLM\SYSTEM\CurrentControlSet\Services\HTTP\Parameters\MaxConnections), the upper limit on inbound connections is 2,031,616. The outbound port connection limit is 5000 by default, the upper limit is 65,536 and is set by updating another registry key (HKLM\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\MaxUserPort).

I hadn't even considered this aspect, thank you for bringing it up. I assume the above is in relation to the server that the JS widget is hosted on?