Working with Site Data in Node.js

Data from users

Another technique that should be familiar to you if your background is classic ASP or MVC is form posting. With this approach, it is assumed that there is a screen into which a user is entering one or more fields and that the entire collection of those values needs to be quickly and easily transported to a URL. To implement form posting, instead of using a get function inside of your .js file, you need to use a post:

router.post('/survey', function (req, res) { });

As you can see, post is almost identical in signature to get. Before we get more deeply into it, just touching on the subject of post as an alternative to get leads us back into taking a brief look at the four basic actions universally available over an HTTP web connection:

• Get A typical web request
• Post Usually used for sending a collection of data to be processed
• Put Usually used for updating a single record
• Delete As you might have expected, usually used for deleting a single record

A file that contains routing information will typically contain at least one of each of the four methods just defined, like this:

router.get('/survey', function (req, res) { });

router.post('/survey', function (req, res) { });

router.put('/survey', function (req, res) { });

router.delete('/survey', function (req, res) { });

As you have seen, often more than one get function exists within this collection. Within the post will be the code to get the name/value pairs from the inbound form collection. Once again, the key to this is having the proper npm package installed. In this case, it is body-parser. So, in your app.js file, make sure you have this line:

var bodyParser = require('body-parser');

You need the variable because you also need to place this line in the same file after the bodyParser declaration just shown to properly format the inbound input:

server.use(bodyParser.urlencoded({ extended: true }));

With those pieces in place, from inside your post function, you’ll be able to access the body property of an inbound request by doing this to get your hands on the value you seek:

var sInput = req.body.txtInbound;

In this case, you are looking for a control called txtInbound.

To see this is action, you need to add a few things to your HTML/EJS file to activate a form post. Let’s start with a button and a textbox. Just to do this demonstration, go ahead and drop a couple of input controls in a separate row below the list control you have in the page:

<table class="table">
<tr>
<td valign="top" align="center">  
. . . 
</td>
</tr>
<tr> 

  <td>
<form action="/survey" method="post">
<input id="txtInbound" name="txtInbound" type="text" />
<input type="submit" />
</form> 
  </td>
</tr>
</table>

Notice how you have wrapped your input controls in a form and then specified two important attributes—method=’post’ and action=’/survey’—to tell the form how you want it to behave. When you submit it, you want the form to post its information to the path indicated in the action—in this case, to your survey page.

With all of this wiring in place, let’s turn our attention back to the actual post function inside your survey.js file to have it respond to your successful form post. Again, we’ll do something more useful with this later. Just to see it work, let’s have it write any input value to the console:

router.post('/survey', function (req, res) {
 var sInput = req.body.txtInbound;
console.log(sInput);
res.send('posted');
}); 

At this point, you can pop the page open in a browser. You should see your button control and your text box. Enter any value, and click Submit. You should see the value you typed appear in the console window. Don’t forget to respond with something; otherwise, your application will stop responding even if the code works as expected.

Typically, this is where you have your CRUD (Create, Update, Delete) interactions with an external source such as a database. When you return to this code in the next chapter, you’ll be taking that value and inserting it into a Microsoft SQL Server database.

Now you’ve seen the Node.js versions of standard get and post operations.

However, aside from these basic, good old-fashioned web techniques for moving bits of data from here to there, inside your Node.js application you do have other options. One of the best of these is the cache.

To use a cache in Node.js, all you need is the proper npm package. You already installed memory-cache when you set up our application so now you just have to do the usual to enable its use:

var cache = require('memory-cache');

This component works just as you would hope that it would, similar to the .NET cache but without some of the features. To put a value into the cache, you simply do this:

cache.put('players', arrPlayers);

And to retrieve that value, this is all it takes:

cache.get('players');

This caching component also has an expiration argument, expressed in milliseconds:

cache.put('players', arrPlayers, 5000);

After the time elapses, in this case five seconds, the item will be removed from the cache if it is not renewed.

One of the most important things to notice here is that my example for storing something in the cache uses not just an object but an array of objects. The cache will hold anything you can concoct in JavaScript, and this idea opens the door to the true power of Object-Oriented JavaScript (OOJS) and sophisticated state management that’s required in commercial applications. It’s a pattern used in .NET to take advantage of the full power of the programming model.

The Node.js cache does for JavaScript objects what the .NET cache does for business-layer objects—it makes them universally accessible. There are two kinds of objects typically stored in the cache: lookup objects and user objects.

A lookup object is typically a collection of values that rarely, if ever, changes—something like US state names or US president names. A list of states stored in the database has to be fetched only one time, and then it can be held in memory after that to allow for quicker access from the application. This works quickly and easily because there is no concern that data will get out of sync—in other words, there are no worries that new data will be entered into the database and the cached version of the data will be out of date. With data that never changes, such as US states, that problem is not a problem. A routine that re-created that data once every four or eight years is also not a big issue.

Of course, this design also works for lookup data that changes more regularly. You simply have to account for those changes in memory as well as in the database—for example, by updating the collection in memory on the same button click that allows for a database update. This is one way to greatly improve the performance of your application.

In general, interaction with the database should be avoided except in cases where it simply can’t be, such as for CRUD operations. Most other functions that people typically perform in the database, such as filtering and joining, can be done much more quickly by using server memory.

Picture a software application as the city of San Francisco, and imagine the only four bridges over the bay represent the database. No matter how many creative ways you navigate to one of those bridges, you’ll be slammed with all the rest of the city traffic doing the same. Those are the only routes. So everyone has to use them. If you keep your database interactive operations to the bare minimum required, traffic will flow better all over your “city.” That’s the whole idea behind using as much cached data as you possibly can.

A user object holds all the information specific to a site user. That can be data from the system itself, such as profile information containing permissions, and it can be state data that is tracking current user behavior. Both kinds of information will typically be required all throughout the application, and the cache is the perfect tool to use for allowing it.

Only one difference is required in the way you manage the cache. For a lookup object, this will work:

cache.put('leagues', arrLeagues);

However, for a user-specific object, you need an identifier that ties that specific object to that and only that specific user. The standard technique for doing so is to create a globally unique identifier (GUID) that you associate with that user, most often on login. Then you simply prepend the GUID to your cache entry like this:

cache.put(GUID + 'User', myUserObj);

You should have that GUID for that user included in the QueryString on every request, like this:

http://127.0.0.1:1234/details?GUID=1QW3456tr4RTYUXXYiujjii45UY89898TRFReded

That way, you can then pull it out to get access to your user object in the cache, like this:

var sGUID = req.query.GUID;
var myObj = cache.get(sGUID + 'User');

You have a rock-solid, state management strategy in place that works for every page of your application, with code consistency, in exactly the same way.

As I mentioned, this caching technique is the only truly viable solution for all web scenarios, even in the world of .NET. If you don’t believe it, try to pass a Session variable across a protocol change— that is, take one of your Session values and pass it from HTTP to HTTPS. Good luck! There’s no way that coding technique will ever work. Sessions do not cross protocol boundaries. You can, and want to, create Session equivalents using the login GUIDs, caching, and OOP, but that’s not nearly the same thing as using the Session object.

You can even take the idea one step further for web-farm scenarios by serializing the data in your objects to external data stores. Serialization turns the state of an object into text. So you serialize to store the data and deserialize to retrieve it. When a request comes in, you check the cache in that specific server for the GUID-related user object. If it isn’t there, you pull the user state from the external store according to the GUID in the QueryString and reassemble it into objects right there. And then you are back to normal code operations. One technique, all scenarios, infinitely scalable.

Now you use a tech interview question to separate the wheat from the chaff—no .NET developer worth his salt will ever go near Session. Like form posting, it’s technology from the 1990s, and .NET gave you much better ways to do the same things starting in this century, and it still does. By being well-versed in those best practices, you’re fully prepared to implement the same architecture in Node.js.

At this point, you’re effectively moving your data from page to page. Next let’s connect to some external data and see what you can do with that.