BecauseRobots

Just Relax: Why it's difficult to plan for scaling

Wed, 16 May 2012 20:01:58 -0400

Diablo 3 was released yesterday, and the authentication servers in particular are having trouble keeping up with the load. Parts of the community are being supportive, but other parts are being obnoxious enough that I wanted to rant. So, here’s the rant for any of you interested enough to read it.

Computers are awesome at doing one thing at a time. That’s all they did at first, we just figured out how to make them very efficient at switching between a lot of different tasks.

The ‘speed’ of a processor (say, 3 Gigahertz) refers to the clock rate of the processor. This doesn’t exactly translate to how many instructions a processor can perform per second, but its closely related. (This is affected by other things, not the least of which is the architecture of the system itself.)

These days, most home systems have multi-core processors. Here, each core can do its own instruction at the same time, So that unrelated ‘tasks’ can be done in parallel. High end computers often have multiple processors, each which can have multiple cores. Something like the authentication system for Blizzard is almost certainly composed of multiple servers.

Problems don’t arise from CPUs not being fast enough or there not being enough servers.

If you wanted to design a car to go 1000 miles per hour, you’d might start by building a car that can go 10 mph and test it up to 100 mph. You’d be able to see things like whether your engine keeping up or whether the wheels stayed attached, but you probably would never discover that at 300 mph, there’s enough upward force generated by the air moving under the car that it’ll lose traction and flip over. The point is, some problems, even with computers, you can’t predict by simulating anything less than full load.

Now pretend you’re Blizzard. You have a new game with over 2 million pre-orders alone. Do you try to simulate a million people connected? How would you even know that you’ve properly simulated a million computers, each with different hardware and operating systems in a way that’s realistic? Or do you run betas and try to find all the serious issues with a hundredth of the real player base, and try to plan to scale from that?

Lastly, remember that this login system isn’t just returning a list of online/offline. Its loading up characters, progress, and even dealing with trying to verify your account is valid and paid for and you’re not some kind of hacker trying to break into someone else’s account.

So, let them have some slack. It’s a good game. If you have to, pretend that the real launch isn’t for a week or two and play it then; there’s no subscription you’re wasting, you’ll be able to play this game now as it is or you can wait for the fixes. Either way, hopefully if you bothered to read all this it offered a little food for thought about why this is hard.

OAB-Java6 – running Oracle and 32-bit Java on 64-bit Ubuntu Systems

Wed, 16 May 2012 13:50:30 -0400

This was a huge time saver, and was hard for me to find after the switch to OpenJDK, but it works great:

https://github.com/flexiondotorg/oab-java6

Highly recommended. I needed this (well, didn’t need but I have a GUI again) for Juniper Network Connect (along with the script on mad-scientist.us.)

Overload – customize your commands

Tue, 15 May 2012 14:13:29 -0400

I just pushed Overload to GitHub. A development version (with even less documentation) was on Bitbucket for a while, but I had made changes and wanted to share the script where other people could find it. (Plus, hg-git has helped me be happier with using GitHub since I can keep the hg client, at least for now.)

The philosophy behind overload really came down to namespacing and discoverability. Over time, I both write and accumulate scripts to perform various tasks. Often the scripts are very small, but other times, say, when I want to use my colleague’s svn switch script, my ‘path namespace’ was feeling cluttered. Overload is my way of being able to not necessarily keep everything in the path and group related actions together.

My next goals for Overload would be to setup some better bash completion, so that it’d be simple to define bash completion for functions that don’t do that yet, or extend their completion to included any commands added through Overload.

Poor Code is Always Poor Code

Tue, 08 May 2012 22:30:58 -0400

I just read an article written by Bejamin Sandofsky that I was sent by a friend that I wanted to respond to. So, here it is.

Poor programming yields poor results, regardless of your language or platform. Some languages simply are not capable of certain kinds of functionality or have been extended poorly. I would never recommend trying to implement real time 3D graphics in HTML/Javascript. I can’t imagine using Java to display content markup (at least not without some kind of embedded browser.) It is important to identify what you’re trying to do and comparing it against the capabilities of what you’re working with.

I helped develop the PetWatch Mobile App at iMarc, which ended up in the app store five or so months ago. Before beginning development, we looked at Sencha, PhoneGap, Titanium, as well as developing in Objective-C. We were interested in other platforms, but were only targeting iOS. We chose PhoneGap.

It isn’t perfect; I certainly could have spent a lot more time going through and polishing up transitions, working on Android support, and adding multi-touch support. Still, I am happy how the UI interactions turned out.

I spent a lot of effort making sure to get the interface for manipulating animals to work as I hoped, using CSS3 and JavaScript to allow me to rotate and scale animals simultaneously around their far corners. The end result feels natural enough that it doesn’t really require explanation to use.

I also spent some time focused on the tray at the bottom that animals and borders are chosen from. I wanted to be able to emulate that iOS style tray that native applications use, and even went as far as to duplicate momentum.

I do agree with Sandofsky that one of the worst parts of developing this mobile app was dealing with PhoneGap. I did end up writing some Objective-C, either tracing bugs or extending plugins to be able to account for the minor, system level integration we needed to send emails with attachments or access the camera. However, that’s about the extent to which I’ll agree with him.

We did run into one issue with how HTML5 was implemented on Android, however that was it to speak of. For this kind of development, we could target solely webkit-based mobile devices, and there were almost no differences. There was an issue where 2.3 Android didn’t implement the HTML5 Canvas elements toDataURI method at all. We also noticed a huge difference in the number of touch events triggered on an iPhone and on Android. The iPhone 4, for example, triggered nearly twice as many events per second during a drag event than an equivalent Android phone like the Motorola Droid 3 or even EVO 3D. This allows motions on the iPhone to be tracked much more accurately, which not only helps with animating dragging more smoothly, but also made calculating momentum much more accurate.

Without devolving too far into a rant against Apple here, I think the belief that there’s a single convention that all native mobile applications follow is ridiculous. After reviewing a number of UIs as we designed the layout for PetWatch Mobile, we found top tabs and bottom tabs. We found paged trays and weighted trays. We found some conventions for some symbols, but also found lots of applications that simply used their own. Certainly once we looked cross-platform, we found even the same app on different platforms would use significantly different layouts and conventions.

We all seemed to agree that we had chosen the right behaviors, even if we weren’t sure on all of the influences. Certainly details like the back/next buttons on the top right and left are trying to match an iOS pattern. I’m not sure if I’ve found anything I’d credit as the source for teaching me that scale and rotate should happen simultaneously around the far corner, but even through user testing that was clearly the right approach.

While I understand that Sandofsky was primarily attacking shell apps, all of the platforms we looked at implemented a way to do some kind of local storage without any issue. Even with only HTML5, local storage was available. We’ll use the app store to deliver new content, as we can release content as quickly as we want and still allow users to get updates at their own pace without any risk to mixing versions.

Lastly, HTML5 is easier. I’ve written Java and worked with Eclipse/Equinox as well as SWT, written Objective-C, and written lots of HTML. While I might be able to create basic interfaces just as easily with any of these, the development cycle is much slower for design outside of the web. Recompiling the PhoneGap app and installing it, even on an emulator, took significantly longer than developing in a browser where I had much richer development tools like Inspector and Firebug. 90% of the PetWatch Mobile application was developed in browsers without any PhoneGap dependencies.

Most importantly, the copy, look, feel and images all could be developed in browsers by web designers without a Java/Objective-C developer to get in the way. While Sandofsky or I may be capable of creating frontends using these other languages, now any designer who wants to actively develop the user interface is really at the mercy of developers. I expect that this is often what leads to such poor UI design on low budget or open source native applications, where the time and budget given or for a project just simply can’t afford the number of iterations that would be required to get to the same quality of design that most web applications or web sites do.

One of my favorite applications to mention in this context is League of Legends, a independently developed computer game first released in 2009. Reported (at least at this time) to have over 500,000 concurrent players, 32 million registrations, and doubling its player base every four months, Riot Games seems to be doing quite well. (1, 2.) And, the game’s out-of-game UI is Flash/Flex based.

The out-of-game UI (the UI you use while outside of any match to chat, view profiles, purchase in game content and form groups) is one of my favorite UI designs I’ve encountered. While It certainly could use some tweaks here and there, it acts exactly how I’d want it to behave: like any other window.

Especially on the desktop, we’re used to having windows that act like windows, hyperlinks, the ability to minimize or resize, etc. Most games simply take over the monitor, filling it from edge to edge with whatever art they can up with, even if there’s nothing deserving this attention going on. League of Legends acts more politely; switching to fullscreen when the content demands it, and being a unobtrusive, light weight Flash/Flex app the rest of the time.

Most poor applications these days aren’t poor because they’re written in Java, HTML, or Objective-C; they’re poor because they’re written poorly. Even most speed issues that people run into with performance isn’t because any of these technologies in inherently incapable of not running fast enough, but because there’s usually flaws in design which cause things to run excessively or redundantly. Scripting languages from Python through Ruby through even HTML/CSS/JavaScript are more concise, often easier to read, and therefore are usually easy to audit for unnecessary looping or excessive constructions. Lower level languages like Objective-C, Java, or C# will always be faster when you really do need to find a way to execute a function as fast as possible, but that’s why extensions to most scripting languages are written in these. Projects like Jython are trying to integrate the two together so each can be used where appropriate. Still, 95% of the time, the code we’re writing, especially for mobile apps, isn’t about the speed of execution; its about making sure we have readable, maintainable code that’s cheap to update and quick to audit.

The only way poor code gets better is being accessible, easy to share, and easy to follow. Choosing Objective-C so you’re poor code can be executed slightly faster isn’t the solution; writing your code well enough you can figure out what’s wrong with it is.

Why you should be using Bash Parameter Expansion

Mon, 07 May 2012 20:01:54 -0400

One of the less commonly used parts of Bash is its parameter expansion. It allows you perform simple string operations on variables without affecting them. For example,

 for F in *.jpeg; do mv $F ${F%.jpeg}.jpg; done

Will find and rename all of your .jpeg files to .jpg files. There are ways to chop off bot suffixes and prefixes, as well as replace strings. For example, I could have used this:

 for F in *.jpeg; do mv $F ${F/%jpeg/jpg}; done

Bash Patter Matching syntax is very powerful, although it differs a lot from PCRE and most other regular expressions. For parameter expansion, the ‘%’ after the first slash is forcing the pattern matched to be at the end of the string, so even a filename like cool-jpeg-I-wanted-to-share.jpeg would get renamed properly.

Unfiltered

Sun, 15 Apr 2012 22:48:00 -0400

I liked social media more when people only used to talk about themselves. I joined Facebook in October 28, 2004; seven and half years ago. Back then, Facebook pretty much was for sharing links to newfangled websites “YouTube” or telling the world whether you broken up with your boyfriend. Now, Facebook seems to be inundated with media to try to change my opinion, and it’s time we stopped. Here’s my opinions, which I’ve tried to make as ‘opt-in’ as possible.

Today’s politics are more about manipulating others than working towards the future. Nothing seems too promising, but at the moment, Obama seems to be trying out some new tactics which at the minimum, are changing the game, probably for the better.
Today’s religions are more about maintaining influence than improving the world. I think there are inherent flaws in taken ancient texts literally; it’d be equivalent to interpreting Beowulf to mean we should all be hunting dragons. Further, even if a higher power existed, they would want us to prove that we were capable of independent thought and reasoning, just as a parent would wish that their children would be capable of making their own decisions.
Today’s morals are more about political or religious motives than quality of life. Today’s moral issues like stem cell research, privacy, or abortion are so obfuscated by other topics that actual discussion about what’s best for humanity is out of the question, even if these are the topics that should be the most important to us.

To be clear, I don’t care if you have opinions different than mine, or that you support or share them, but we need some moderation. Facebook is swamped with these kinds of opinions and I find that I tune Facebook out as much as I tune out mainstream media. I have as much interest in what some friends post to Facebook as I do with what news story CNN or Fox just added to their ticker.

I think the core issue is that its easier to tell other people to care than to care yourself. It’s easier to tweet than donate. It’s easier to post to Facebook than it is to vote. Social media is devolving into either attacking or defending how other people should live, drowning out any meaningful communication left.

Because there is no ‘Filter Politics’ option on Facebook. I can only choose to filter everything you say. Next time you decide to repost some witty pro-choice article or heart-warming story of God Saves, think about whether the people you’re sending this will care.

Don’t resend chain emails to everyone in your address book without thinking. I guarantee that if you spent less than 10 seconds thinking about what you’re doing, I’m going to spend less than 10 seconds paying attention to anything you say. Eventually, if you do it too often, I’ll find a way to not pay attention to anything you say.

Believe anything you’d like. Support it through actions or donations as strongly as you feel and are capable. Publicize it only when you’d spent time thinking about what you’re going to publish. Otherwise, when you do cry wolf, no one will care either.

More Site Design

Sun, 15 Apr 2012 21:31:00 -0400

So, I’ve continued to tweak this Tumblr skin. I’m reworked my navigation, improved the footer (and added footer navigation), and added media queries to load different styles for small screens and mobile. The site should function (albeit that there’s a lot of JavaScript errors from SHJS in ie8 and lower) as far back as IE6 without being entirely broken, and gracefully improve all the way up. Since the mobile site relies on media queries, you won’t see the thinner version by resizing your browser unless you’re running at least IE9 (or Chrome/Safari/Opera/Firefox.)

Any opinions about the new look would be most appreciated.

Remapping Caps-lock from a Terminal

Tue, 10 Apr 2012 13:11:00 -0400

After playing around with xmodmap and some other programs, I finally found setxkbmap which allows you set an XKB setting. There’s loads of settings (for me, they’re all listed in /usr/share/X11/xkb/rules/xorg.lst) but here’s some highlights:

Make caps-lock into a

control key: “ctrl:nocaps”
control key, and the control key into caps: “ctrl:swapcaps”
escape key: “caps:escape”
super (windows/command) key: “caps:super”
compose key: “compose:caps”
make caps-lock do nothing: “caps:none”

To run of of these, use setxkbmap like this:

setxkbmap -option "caps:super"

That’s all there is to it.

Using JavaScript to create JavaScript-less, CSS Animations

Thu, 23 Feb 2012 20:03:00 -0500

First off, demos: you can see a nice, webkit-only jsFiddle of a slideshow or of a ticker.

Note that this isn’t widely useful yet, as CSS animations aren’t widely supported. Still, I thought it was worth pointing out this logical difference.

I created some simple ‘slideshow’ style animations but unlike most, JavaScript execution stops entirely after running through once.

Similarly, I made a ‘ticker’ that stops moving on hover, that’s created by running JavaScript once, and using only CSS for the animations (even the pausing.)

The New Keyboard Layout is in Progress

Thu, 23 Feb 2012 13:33:00 -0500

I’d say, all things considered, I’m probably two or three days away from ordering my custom keycaps. Almost all the buttons are there and I’m really just culling the worst of them and trying to come up with even better ones. My second Leopold did arrive, so I have that now. I also let @bratling (who I work with) try the keyboard out, and he’s thinking about ordering one in the next month or so, though perhaps with browns.

I did hear back from WASD Keyboards that I could get a full set of custom keys easily enough, so everything seems in order. I am not sure if I will order both sets of keys immediately or not; I’ll have to wait and see what happens. After reviewing a number of fonts, I think I’m going to use Terminal Dosis for the lettering since its one of the best free fonts I could find that really had the right kind of retro feel to it.

Custom Keyboard Keycaps

Wed, 15 Feb 2012 02:24:25 -0500

Just bought myself a second Leopold keyboard to match the one I have at work (tenkeyless, Cherry MX Black keyswitches.) I thought very heavily about getting a WASD keyboard instead, but decided that I’d rather have two matching keyboards instead.

After seeing on WASD’s blog what they can do with custom keycaps, I sent off an email to see if I can’t work something out to get a set or two for my Leopolds. Hopefully…

More tweaks and a long weekend

Sun, 12 Feb 2012 23:10:21 -0500

I’ve gone back through and retagged most posts for consistency, as well as made some more minor changes to the look. I got highlight.js working once again, which was one of the biggest reasons I ditched Posterous. I even went through and hacked together a custom theme for highlight.js based upon my ambient color scheme for vim. It might not be perfect, but its certainly better than nothing.

I also spent some time this weekend working with Vundle, which is a bundle/addon/script manager for vim that has a few features that pathogen doesn’t; most importantly, a way to search for them, install them, and keep them up to date. I’ll probably right up my thoughts on that in a separate article though.

Other minor things: switched to Ubuntu’s font instead of using Roboto, as I found it looks better and has more the style I wanted to go with. Also, discovered that upgrading VirtualBox can be dangerous if you have any virtual machines in a saved state; it completely corrupted mine and I had to make a new one.

It's a new look

Wed, 08 Feb 2012 22:50:27 -0500

So, I’d say the new look for BecauseRobots is probably about… 80% complete now. This is probably the third major revision of the new look, which is inspired by Android’s style guide and Holo Dark. While a lot of Android’s style guide is clearly oriented for touch devices, I wanted to steal some of the look and feel. They are no where near as detailed or thorough as Apple’s (PDF), but I’m glad to at least see Android taking steps to unify the platform’s appearance and make the experience more cohesive overall.

After going through this process a number of times, I did end up starting with typographical styles and building “mobile first” styles and then worried about any tweaks I needed to make to the layout to look better on desktops. Part of the last 20% that I haven’t done is probably switching the font sizes based upon my guesses at pixel density. I did some of the work for this for the PetWatch app, but there I had the added benefit of exclusively targeting the iPhone 3GS, iPhone 4, and iPhone 5. In he wild, I’ll have to guess screen sizes, whereas I could treat all platform screens as having the same physical size.

I think the real solution will be a way to actually specify layout/sizes in better units than pixels. Android calls them density-independent pixels (dp) and scale-independent pixels (sp), which are terrible names.

In spirit, a dp is a ‘replacement’ for pixels, and what Apple (and others) secretly pretend pixels are based upon meta tags. a 1dp thick line is meant to be a (more or less) the thinnest line that can be displayed on the screen, even if that means being 2 physical pixels thick. This probably what should be used for defining borders and other UI elements.

In contrast, a sp is another ‘replacement’ for pixels, identical to dp except that sp scale with the user’s zoom preference. So, I’d expect that images and other content should probably use sp’s for dimensions.

Anyway, this version of the site doesn’t use that (at least not yet.) Maybe I’ll sweep through and change over.

Energy Drinks Ordered by Taste

Tue, 07 Feb 2012 14:32:00 -0500

A week or so ago, we bought a number of energy drinks for the office to see which ones we liked. Here’s the order from best to worst taste. For reference, I think most soda would fall around a 2 or 2.5 on this scale for me.

Energy Drinks by Taste

Rockstar Black (Double Strength) - tasty, good kick. I know that I also like the white (sugar free) ones as well, but we didn’t get those for the office yet.
Red Bull - tastes like Spree candy in a good way.
Monster - not bad a little more bite than most like.
Moxie - tastes like lemon juice and bug juice like you’d get at camp.
Full Throttle - like if you melted down skittles and mixed them with sprite.

Vintage Mode in Sublime Text

Sat, 28 Jan 2012 14:10:00 -0500

Will is one of my colleague at iMarc, and an avid Sublime Text 2 user. He’s written a number of plugins for it now, and every once in a while vintage mode comes up.

Vintage mode is an ‘official’ plugin that’s bundled with ST2 itself which tries to add many vi behaviors to ST2. The first couple times I tried it, I found short comings within 2-3 minutes and this time it took 5, so it’s slowly getting better. Anyone who had used vi or vim (or gvim/macvim) for a while will probably notice these.

These are all command mode or visual mode commands, as that’s all Vintage mode claims to affect. There’s certainly a number of ex and insert commands which I would sorely miss as well.

The lists below are by no means exhaustive, just commands that I use fairly often and recalled from memory.

Vim Left-Right Motions

h, l (left and right) - already found differences. h and l don’t ‘wrap’ around the beginning and the end of the line in vim, and in Sublime, they do. If you think this is an irrelevant difference, think of using multi-selections to edit a bunch of lines at once. vim is very mindful of what commands will cause line changes and which ones won’t, and I think this ties into making sure that macros are more robust or degrade more smoothly.
0, ^, $ (first character, first non-blank character, last character) - these all seem to be consistent.
g^, g$ (first character of the visual line, last character of the visual line) - these don’t seem to exist. It’s interesting that Sublime implemented the ‘true’ line operations here and didn’t implement the visual line operations, whereas in other places, Jon did the opposite.
f{char}, F{char}, t{char}, T{char} (go to the next/previous character, go up to the next/previous character) - these all seem to be working correctly.
; and , (repeat last f, F, t, or T command) - all seem to work.
{column}| (goes to column {column} in the current line) - not implemented.

Line Motions

j, k (up and down) - in vim, j and k do their best to move to the same column in the previous line. In Sublime, j and k move to the same visual column and may only change visual lines. The ‘visual line down’ and ‘visual line up’ commands do exist in vim too, they’re just gj and gk respectively.
gg, G (first line and last line) - seem to work.
+, - (goes to first character of next/previous line) - don’t exist.

Word Motions

w, W, b, B - these all seem consistent. For those that don’t know, w moves your caret to the next first character of a word (like ctrl-right/cmd-right.) Similarly, b moves your caret to the previous first character of a word. W/B are nearly the same, but they work on WORDs instead of words, where a WORD in vim is a string of any non-whitespace characters.
e, E, ge, gE - these are very similar to w/W/b/B, but instead of going to the first character, they go to the last character of a word or WORD. e/E seem to work fine, but it looks like Sublime is lacking ge or gE entirely.
cw/cW - this is an inconsistency in vim (maybe even a bug originally), where cw should have included the space(s) after the current word, but instead, cw and cW were changed to behave identically to ce and cE. All things considered, its extremely rare that anyone would have actually wanted the correct behavior, but it is still a little strange, as similar commands like dw and vw do not behave the same as de and vw. Still, Sublime decided to implement this to be consistent with vim, so all is well.

Text Selection

viw,vaw,vaW,viW (select word/WORD without or with whitespace) - these seem to work.
vit, vat (select inside tag, inside tag and tag) - these don’t seem to be implemented.
vi’, va’, vi”, va” (select inside single/double quotes without or wite whitespace) - these seem to work.

I didn’t test clipboard/registers very much, but I did want to note that while it appears that dd (delete line) works fine, pasting the line back doesn’t correctly preserve newlines. ddp should just swap two lines, whereas in sublime I have to ddpi<enter><esc>. Another nicety would be for Vintage mode to change some global keybings, like <ctrl-r> to redo.

Bring skates.

Thu, 19 Jan 2012 23:14:00 -0500

Bring skates.

It's in Progress

Tue, 17 Jan 2012 23:12:00 -0500

Just wanted to let everyone know that my tumblr theme redesign is in progress, so the site probably looks messy at best. I’m trying out some fun things, like jQuery Masonry, to try to get a unique style that isn’t too terrible.

Also, I’m trying to make sure all my fonts are bigger so the site is easier to read, and trying to update the font. Right now I’m playing around with Segoe UI and Segoe UI Light, but I may also try something more interesting.

Anyway, thanks for the patience.

'Disabling' the F-Lock key on Logitech SetPoint Keyboards

Sat, 14 Jan 2012 12:55:00 -0500

It took way too much Googling to find this obvious, simple solution, so I figured I’d post it to BR where it’ll hopefully be easier to find.

In SetPoint, under My Keboard, Keyboard F Key Settings (the F in a box on the left) you can select what all the F keys do when F-Lock is on. Just go through and change each to Keystroke Assignment and assign them to their respective function key. While this doesn’t disable F-Lock at all, it merely makes the function keys behave the same regardless of whether F-Lock is on. This works perfectly with my old diNovo Media Desktop keyboard that I use for gaming.

nslookup cobblestone

Thu, 12 Jan 2012 01:27:00 -0500

I am overtired and going to sleep now and wanted to share how I figured this out.

While playing Minecraft, my mind started to dream anyway, and I dreamt that there was some kind of DNS issue into of Minecraft, and I was confused because I wasn’t sure how I figured out there was a DNS issue as I didn’t see any trace routes or text on the screen. It was very confusing; I

Good night.

PHP: Joining arrays into subsets

Wed, 11 Jan 2012 16:21:00 -0500

The following snippet joins the elements of an array into of subgroups of 3 (and one for the remainder.) I used this for grouping list items into columns for formatting, if it helps, enjoy.

$x = array('one', 'two', 'three', 'four', 'five', 'six', 'seven', 'eight');
for ($i=0; $i<count($x); $i++) {
    array_splice($x, $i, 3, join(array_slice($x, $i, 3)));
}

Creates

array(3) {
  [0]=>
  string(11) "onetwothree"
  [1]=>
  string(11) "fourfivesix"
  [2]=>
  string(10) "seveneight"
}

By the way, apologies for the crappy formatting for code snippets; I’m working on a new style for BecauseRobots which should also address that.