• About This Newsletter
• What Are You Thinking? => The JavaScript Book
• On the Web => Improving Application Performance
• On the Web => Opera Dragonfly
• On the Blog => UTC and Time Zone Support in MySQL
• On the Blog => How I Cut My Bandwidth in Half
• Q&A => Should I use PHP or JavaScript to Handle API Data?
• What is Larry Thinking => Doing What I Do: Training
• Larry Ullman’s Book News => “PHP and MySQL for Dynamic Web Sites” (4th Edition) and More!

About This Newsletter

No overarching theme to this newsletter, just some things that have popped up over the past three weeks, plus a preview of my next few months with respect to the books I’ll be writing. This newsletter also continues my spontaneous series on pursuing careers in IT, specifically focusing on training and teaching.

As always, questions, comments, and all feedback are much appreciated. And thanks for your interest in what I have to say and do!

What Are You Thinking? => The JavaScript Book

If you’ve been reading this newsletter for a while, you’re probably familiar with, if not sick of hearing about, my intent to self-publish a book on JavaScript (I’ve posted a rough Table of Contents and description online). Part of the reason for self-publishing this particular book is that most every publisher already has their own JavaScript book already. Another part of the reason was to have the most control over the final product, both in terms of the content and the destination formats. I may or may not make more money self-publishing, so that’s not really a mitigating factor (I’ll make more per copy sold, electronic or print, but wouldn’t likely sell as many copies as I would through a publisher).

I’ve already started writing the book and done some research with respect to self-publishing, all with the hope of releasing the book some time this summer. In a surprising turn of events, a publisher has come to me wanting to put out the book. The publisher is very much willing to work with me on the concerns I have about control and the quality of electronic formats. And the publisher is better equipped to get the book into more places, including various electronic and online resources, international destinations, and translations. I’m clearly being coy about some of the details, but my question to those of you that might be interested in a JavaScript book written by me is whether you have a strong preference between a self-published book and one that’s traditionally published. Does it make a difference to you? Once you set aside the issues of subject and writer (which would be the same in both cases here), what criteria do you have for whether you buy a book and for how much you like a book?

As a writer, I have an entirely different sense of these things, so I would greatly appreciate any feedback you may have on this particular issue. I know which way I’m leaning here, but want to make sure I’m not overlooking something important. Thanks!

On the Web => Improving Application Performance

I’m a big fan of Adobe AIR, and not just because I’ve written a book on it! I’ve been developing AIR applications, for in-house and client use, for a few years now, in both JavaScript/HTML and Flex. I’ve recently, and not so recently, come across these two good articles on Adobe AIR:

• Performance-tuning Adobe AIR applications
• Ten tips for building better Adobe AIR applications

From the titles, the articles are self-explanatory, but the emphasis is on performance. The interesting thing about the first article is that it discusses the theory of performance, which people don’t think about enough (e.g., what does it mean to perform well?). The article uses a specific example for which one could easily come up with three different senses of “performance” (the article uses Flex for the code).

The second article has lots of specific, excellent tips, many of which being applicable to any application you develop (although the example also uses Flex for the code, it’s mostly ActionScript that you’ll see).

Even if you’re not using Adobe AIR, I would think these articles would be worth reading, as the subject of application performance is one we could all always continue to learn more about.

On the Web => Opera Dragonfly

Admittedly, Opera is about the fourth browser on my list (after, in order, Safari, Chrome, and Firefox), but the release of Opera Dragonfly may just change that for me, at least when it comes to Web development. Historically I’ve always relied upon Firefox’s Firebug extension when I need a good development debugging tool, but the Firefox browser has become such a memory hog for me that I’ve stopped using it on a regular basis. Of course, it’s probably a memory hog exactly because I’ve installed so many extensions (Firebug and the Web Developer Extension and …)!

On a cursory glance, Dragonfly may have everything I need:

• DOM viewer
• JavaScript debugger: allowing you to set breakpoints, inspect the values of variables, and set conditions to watch for
• HTML and CSS editor
• Resource inspector: what resources are loaded by the page
• Network traffic tool: a must for debugging Ajax, including the ability to craft your own requests to see the response
• And more

Unlike Firebug and other extensions, Dragonfly is built into Opera. Also unlike any comparable tool, Dragonfly can be used for remote debugging, including content destined for mobile devices or televisions!

I don’t know that Dragonfly alone will make me switch to Opera as my primary browser, but I have a strong feeling it will be my primary development browser for some time.

On the Blog => UTC and Time Zone Support in MySQL

In the third edition of my “PHP and MySQL for Dynamic Web Sites: Visual QuickPro Guide” book, and now in the fourth, too, I discuss using Coordinated Universal Time (strangely enough, abbreviated UTC) for storing dates and times in a MySQL database. I ran into a hiccup installing support for time zones in MySQL on my Mac, so I recently wrote a short blog post introducing the subject and detailing the installation particulars on both Windows and Mac OS X.

On the Blog => How I Cut My Bandwidth in Half

Every so often I take a look at the Web stats for my site, as I expect we all do from time to time. It can be addicting stuff. Looking over the numbers in May, I realized that I managed to cut my bandwidth in half (since January), without ever intending to. In this blog post, I write about the steps I took that achieved this result, even though that wasn’t a goal.

Q&A => Should I use PHP or JavaScript to Handle API Data?

Daniel, who asks many good questions for use in these newsletters (thanks!), posed this:

Can I use PHP or JavaScript to look through an array or parse data from a Web Service API in XML or JSON?

The answer, in my opinion, depends upon what you intend to do with the data. If the data is needed temporarily, for example, to show something in the Web browser, you can use JavaScript. The benefit to JavaScript is that it will be the client (i.e., the Web browser) making the request, not the server, which can improve performance and reduce the server load. The downside is that JavaScript must be enabled in the browser and there are restrictions/tricks to performing cross-domain requests using JavaScript.

If the data will be needed for a longer period, if the same data might be presented to multiple users, or if you must do complex manipulation of the data, I would recommend using PHP. You’ll most likely want to use the cURL library in PHP to do this, meaning your PHP version must support cURL and you’ll need to learn how to use it (although cURL isn’t that complicated and the PHP manual has good examples).

As with any time using an API (Application Programming Interface), I would strongly recommend you take the time to read the API’s usage terms to ensure compliance. For example, it may not be legal for your site to store data retrieved through an API, which would be another factor to consider.

What is Larry Thinking? => Doing What I Do: Training

In this newsletter I’m continuing what became a series on IT careers. I first wrote about becoming a better programmer, in two parts(1 and 2). Then I wrote about building a career and how I got here. In the previous newsletter, I wrote about some of the specifics of what I do, focussing on the writing side. In this post, I’ll discuss another facet of my daily job: training. Unlike in the last newsletter, I’ll be more general about money here, as there’s just too much variation.

In the previous article, I said that about 60-70% of my income comes from writing, with the other 30-40% from training/teaching and Web/application development. In terms of training and teaching, early on I taught an online course on PHP through the University of California at Berkeley. Much like writing, it’s the type of work that sounds more impressive than it pays! I eventually stopped doing it, as it was a lot of work for not that much money, with quite a bit of responsibility and hassle. If you’ve done any teaching at all, you know of what I speak. But should you want to pursue this route, the best way to get such jobs is to already be associated with a school in some capacity (i.e., have an “in”). There is an argument to be made with this avenue that you’ll have somewhat regular income, albeit small, and it’s the kind of thing that you can perhaps use as a selling point for getting other work.

Off and on for the past several years, I’ve done more training jobs, by which I mean corporate on-site training over one or more days. Training jobs can pay very well, but are hard to get. If you’re interested in going that route, you should probably hook up with an organization that manages training. For example, MySQL arranges training courses, so you could work with them (I don’t know how the arrangement has changed with Oracle’s purchase of MySQL, though). There are also companies that do all sorts of training, helping to market the courses, provide the facilities, and so forth. To work with one of these organizations (I’ve had discussions with them myself), it’s just a matter of proposing a class that appeals to them and extends their catalog. Much like trying to get a book or article published, it’s a matter of bringing a good idea to the right people. If you can get involved with training, and don’t mind traveling, you can make a very good living doing this. I personally consider it to be stressful, but I am deathly afraid of public speaking and put a lot of pressure on myself to have all the right answers going into such sessions. I still do training occasionally, though, as it gets me out of the house and pays some bills.

(As an aside, speaking at conferences is another kind of thing that sounds great but pays very poorly. In fact, at the entry level, speaking at conferences just earns you free admission to the conference. At the higher levels, perhaps your travel will be covered. But conferences are great networking opportunities and being a public speaker can, again, be a way to find other work, establish qualification to do certain work, and perhaps even allow you to charge more for other work.)

Whether you’re doing training or teaching (or speaking at conferences), do be mindful of who owns the copyright for the material you put together and whether the sessions can be recorded (and, if so, who owns those recordings). In these situations, a lot of your time and effort goes into the creation of the material, which isn’t how you will really make money. The true pay comes from actually teaching the course or providing the training. Make sure you’ll get your due for the work you put in.

I haven’t yet pursued this direction at all, but clearly the big push these days (or these past couple of years) is towards online video training. There are many companies, such as Lynda, that provide this service. To get these jobs, I expect, it’s a matter of convincing the right company that you’re the right person to create a training series on the right subject (like getting any other training job). I believe the economic model for these is a set sum to produce the video materials and then you’ll receive payments based upon usage (i.e., upon people watching them). Like with writing, you can help your cause in this area by producing your own low budget video training sessions to demonstrate your abilities. Post the videos you do on your own Web site and on YouTube, pay attention to the feedback you get, and build a career!

Larry Ullman’s Book News => “PHP and MySQL for Dynamic Web Sites” (4th Edition) and More!

May was a rough month for me, work-wise, which is to say I didn’t get nearly as much done as I would have liked. Now, I knew going into the month that that was going to be the case, but still… Anyway, I’m hoping June will be better. I am a couple of chapters away from completing the first draft of the fourth edition of my “PHP and MySQL for Dynamic Web Sites: Visual QuickPro Guide”, which puts it to be released at the end of the summer. I think the book is turning out well, with nice updates and added content.

I’m probably going to use July as a catch-up month, although I have a number of side projects to work on, just nothing with a deadline. Either in July, or immediately thereafter, all of my attention will go to the JavaScript book (déjà vu: I think I’ve said this a time or two before, but this time I really, really mean it). With as much as I’ve spoken about this alleged and wily JavaScript book, and as many readers have said they’re looking forward to it, I’m starting to get that dread of living up to expectations. But I’ve got a lot to say about JavaScript, I’ve been thinking about this book for years, and there’s so much that can be done with today’s JavaScript, that I feel optimistic.

In the Fall, I’m still hoping to self-publish a book on the Yii framework, although I may just do an e-publication of that. In 2012, I’m planning on (and the publisher has agreed to) doing the third edition of my “PHP 5 Advanced: Visual QuickPro Guide“.

I’ll leave it up to you to read the full article, but the gist of it is that if you’re using MySQL 5.0.37 or greater and using the command-line mysql interface, you can enable profiling to see exact performance numbers for the queries you run. You can even see the nitty-gritty details: everything MySQL does to run the query, how long each step takes, and even what CPU or memory usage was required.

Between the SQL Profiler and the EXPLAIN the command, you can quickly improve how efficiently your SQL commands execute, which will improve the performance of your Web application as a whole. And, in a surprising result, I found that the stored procedures I created ran much, much faster than straight SQL (this was on MySQL 5.1.37). Clearly MySQL has been taking great strides to improve the performance of stored procedures, which used to be notoriously inefficient.

The tutorial goes on to talk about organizing HTML files: CSS at the top and JavaScript at the bottom. I’ve heard this before (about putting JavaScript right before the closing tag) but haven’t gotten around to testing it myself. It makes sense, though, as the browser doesn’t need the JavaScript to render the output for the user to see, and because the JavaScript code itself probably can’t do its thing until the HTML has loaded. If you keep reading, you’ll learn how to use CSS image sprites to minimize the number of images that have to be downloaded by the browser. My Web site has practically no images, so I haven’t used this myself, but it’s something that can dramatically improve the performance of a site.

• The storage engine(s) used
• How columns are defined
• The existence of proper indexes
• How queries are written and executed

MySQL supports many different storage engines, each with its own strengths and weaknesses. Which you choose isn’t just a performance issue, it’s also one of features, like whether you need to support FULLTEXT searches (and therefore MyISAM tables) or transactions (which means InnoDB). Sometimes you can get away with using HEAP tables, which exist only in memory. There’s no permanent storage with HEAP tables, but the performance can’t be beat.

For the column definitions, the first goal is to use the smallest possible size for each column. This will be a limitation in your application, so the column’s size still needs to meet or exceed the largest possible value to be stored, but, generally speaking, the more data you want to possibly store, the more space will be required for all the data stored. You should also avoid allowing NULL values in columns. This is a factor in both good database design and in performance optimization.

The indexes you establish, and how they are used by queries, is a serious subject that requires a book in itself. To start, make sure you read MySQL’s own guide to how MySQL uses indexes. Second, learn to run your queries using EXPLAIN to see what, exactly, MySQL is doing when it executes that query. Also, of course, only select the data your application will actually need. No need to retrieve information that won’t be used. And always go back and double-check that your queries need all the clauses and references they’ve got. I’ve been known to leave legacy conditionals in queries that have no effect whatsoever.

For more on some of these topics, see the MySQL manual, of course (which has a section on optimization), the PlanetMySQL blog, and this article at DatabaseJournal.com. Also, if it applies to you, you can tweak MySQL’s behavior from a hardware/OS perspective. For an introduction to this subject, with respect to Linux, check out this article by Jeremy Zawodny, an expert on MySQL performance tuning.

• Performance-tuning Adobe AIR applications
• Ten tips for building better Adobe AIR applications

From the titles, the articles are self-explanatory, but the emphasis is on performance. The interesting thing about the first article is that it discusses the theory of performance, which people don’t think about enough (e.g., what does it mean to perform well?). The article uses a specific example for which one could easily come up with three different senses of “performance” (the article uses Flex for the code).

The second article has lots of specific, excellent tips, many of which being applicable to any application you develop (although the example also uses Flex for the code, it’s mostly ActionScript).

Even if you’re not using Adobe AIR, I would think these articles would be worth reading, as the subject of application performance is one we could all always continue to learn more about.

New to me, and a slightly negative reflection on my site, was entity tags (ETags). An ETag is a unique identifier for a resource. A browser (and other software) can check a resource’s ETag to know whether or not to download the resource, somewhat similar to how expiration headers are used. An ETag can be any value that represents the resource and changes when the resource changes. Commonly the server might use the file’s modification time for this value, or a hash representation of the file as a whole (e.g., run the file’s contents through an MD5 function). My server was providing ETags for all but the main PHP page. This makes sense, because PHP pages aren’t like static HTML pages: my main PHP page is updated whenever the database it uses for its content is updated.

One fix would be to have PHP send an ETag header:

header("ETag: \"representative value\"");

As already written, the representative value could be any of the following:

• The modification time of the last database record used by the page
• An MD5 representation of the entire page, including the database-driven content
• An MD5 representation of just the most recent database record that’s reflected on the page.

And the answer I came up with was…just to ignore ETags on this PHP page. Coming up with any of those values would require a decent rewrite of the entire page, something I just don’t feel like doing right now. And, in my defense, my score turned out pretty well after I addressed the JavaScript and CSS compression issue…

Another reason I lost points was because two of the files being sent (the JavaScript and the CSS) weren’t being compressed. Most of my site uses PHP, which uses the zlib library to compress data sent to the browser. So PHP on the server compacts the HTML, it gets sent to the browser in compressed format, then the browser decompresses it and reads it. That’s great and generally recommended, but the JavaScript and CSS pages aren’t being handled by PHP. One solution is to use Apache’s mod_gzip or mod_deflate module to compress CSS and JavaScript files. To do that, add this code to an .htaccess or .conf file:

<IfModule mod_deflate.c>
<FilesMatch "\.(js|css)$">
SetOutputFilter DEFLATE
</FilesMatch>
</IfModule>

That code basically says if mod_deflate is loaded, any file that ends with .js or .css should be ouptut using the deflate filter.

After I took all these steps, I got my YSlow grade up to a 98! I’m pretty pleased with that result. Also, in more concrete terms, if a user accesses my home page for the first time, they perform 4 HTTP requests and download just over 21KB of data. Subsequent requests of that same page require only 1 HTTP request and 3KB of data to be downloaded. That should make a big difference.

To wrap up this thread, I’ll mention two things. First, the YSlow plug-in and report is a very useful tool, but it’s not the only test or indicator of a site’s performance. I do think it’s a great place to start, though. Second, in researching some of these fixes, I came across several other sites where people did the same thing I did (tested their site using YSlow, then tried to address the problems). If you’re interested in this process, you may want to search online to read some of those other articles.

(As a post-script, I will add that one can improve the ETag score by disabling ETags in Apache, strange as this may seem. By doing so, it forces browsers to rely upon the caching headers. So if you can’t or don’t want to get ETags to work, turn them off and they won’t cause a problem.)

As always, thanks for reading and let me know if you have any questions.