Monday, April 12, 2010

Facebook Developer Humor

Facebook image upload error: The photo was either too tall or too skinny.

Facebook caps the maximum image size to conserve storage space, or bandwidth, or both. I attempted to upload an image substantially larger than this, and received this error.

I suspect the wording is a joke by Facebook developers, "You can never be too tall or too skinny."


Saturday, April 10, 2010

Bending the Rules

As of yesterday the iPhone Developer Program License reads:

3.3.1 - Applications may only use Documented APIs in the manner prescribed by Apple and must not use or call any private APIs. Applications must be originally written in Objective-C, C, C++, or JavaScript as executed by the iPhone OS WebKit engine, and only code written in C, C++, and Objective-C may compile and directly link against the Documented APIs (e.g., Applications that link to Documented APIs through an intermediary translation or compatibility layer or tool are prohibited).

The way the new requirement is phrased is interesting. Adobe's Packager for iPhone compiles Flash code to native ARM instructions. Assuming the Adobe tool can produce position independent code, one might consider something sneaky.

void run_disassembled_flash_binary()
{
  asm volatile("ldr r3, [pc, #192]\n\t"
               "mov r1, #3\n\t"
               "str r3, [sp, #32]\n\t"
               "mov r3, #0\n\t"
               "str r3, [sp, #4]\n\t"
               "str r3, [sp, #8]\n\t"
               "mov r2, #24\n\t"
               "mov r3, #48\n\t"
               "add r5, sp, #12\n\t"
               "str r0, [sp, #24]\n\t"
               "str r0, [sp, #0]\n\t"
               "add r0, sp, #12\n\t"
               ...

I make this suggestion in jest. Apple does allow embedded ARM assembly in iPhone apps, but an application consisting of nothing but opcodes produced by a foreign programming environment will not resemble an app compiled from more conventional source code. I very much doubt it would pass the App Store approval process. Though phrased as a technical requirement, section 3.3.1 of the iPhone Developer License is really a business imperative.

Thursday, April 8, 2010

Simple Checksums Considered Harmful

Lets talk about iSCSI for a moment, as a launching point for a discussion about data integrity. iSCSI relies on CRC32 to catch data corruption. CRC32 is a good fit for this purpose, but most previous uses of it had been confined to very low levels of the system and implemented in hardware. iSCSI uses CRC32 way up in the protocol header, where it is generally computed in software. The overhead of computing the CRC is one reason why so many hardware offload adaptors were developed for iSCSI.

Intel recently released a whitepaper describing how they achieved 1 million iSCSI operations per second. One fascinating tidbit is that the CRC32 is no longer a bottleneck. The Nehalem architecture includes an instruction to compute it directly, as part of SSE 4.2. The new instruction is described in the Intel64 and IA-32 Architectures Software Developer's Manual Volume 2A: Instruction Set Reference, A-M. It is on page 3-221 of the December 2009 edition; search for CRC32 in later editions.

CRC32 r32, r/m8Accumulate CRC32 on r/m8
CRC32 r32, r/m16Accumulate CRC32 on r/m16
CRC32 r32, r/m32Accumulate CRC32 on r/m32
CRC32 r64, r/m8Accumulate CRC32 on r/m8
CRC32 r64, r/m64Accumulate CRC32 on r/m64

Thats it. You load words from memory and hand them to the CRC32 instruction. If you were already making a pass over the data for any reason, the CRC calculation is free. Table-driven CRC generation implementations were already fast, but this is even faster.

What does this mean? I think it means weak checksums should no longer be used for anything. Applications which care about data integrity moved to MD5 or SHA1 years ago, but you still see specifications in other contexts written to use Adler-32 or even the venerable 16-bit TCP checksum. Its not appropriate to use these any more. Server CPUs can compute CRC32 for free, and embedded CPUs have long included CRC32 calculation in DMA engines.

Monday, April 5, 2010

Peepshi

Just before Easter weekend Serious Eats published an excellent recipe for Peepshi. I highly recommend it. Their presentation is much better, I am just an amateur at food pr0n.


Peeps, Rice Crispy Treats variety pack, fruit sheets.

Four assembled peep sushi.

Ten assembled peep sushi.

Friday, April 2, 2010

Distribution Schmistribution

Avnet LogoElectronics distribution giant Avnet recently announced it would acquire Bell Microproducts. The deal values Bell at $250 million after paying off debt. Bell's annual sales are north of $3 billion, but electronics distribution is not a high margin business.

Some of the commentary surrounding the announcement expresses a lack concern about consolidation amongst distributors, because the chip manufacturers have firm control of pricing. I'm not sure that faith is warranted: distributors lack the ability to control pricing because there is so much fragmentation in the market. Each manufacturer has multiple channels for distribution and can play them off against one another. As the number of strong distributors dwindles, that power shifts to the remaining middlemen.

Distributors regularly show they will take complete advantage of any leverage they get. For example, chip manufacturers give preferential terms to a distributor who wins the design-in at a particular customer: no other distributor will be allowed to offer better prices to that customer. This is reasonable... until the distie takes it to the next level. Distributors do a great deal more business with contract manufacturers than almost any individual customer. A distributor can legitimately win the design for a particular chip, then request the bill of materials from the CM. In almost all cases the CM will give it to them. The distributor then registers as winning the design for every component on the BOM, even those they had nothing to do with. As a customer you suddenly find yourself unable to obtain better pricing on anything in your product, based on a single deal for one component.

If consolidation amongst distributors gives those which remain significantly more power over pricing, does anyone think they won't abuse it?

Monday, March 29, 2010

The World Wide What?

Spider web laden with dew.

Calling it the World Wide Web is in insult to spiders everywhere. From now on, I propose we call it the World Wide Hairball.


Wednesday, March 24, 2010

Player Piano Torpedoes

March 24, 2010 is Ada Lovelace day, an informal holiday to celebrate the achievements of women in technology and science. I'd like to share a fascinating technology story about Hedy Lamarr. Ms Lamarr was a contract star at MGM during the Golden Age of Hollywood, in the 1930s and 40s. She was also a creative and mathematically talented inventor. Today, we would proudly call her a geek.

From US Patent 2,292,387, by Hedy Kiesler Markey and George Antheil:

"This invention relates broadly to secret communication systems involving the use of carrier waves of different frequencies, and is especially useful in the remote control of dirigible craft, such as torpedoes.

Our system... employs a pair of synchronous records, one at the transmitting station and one at the receiving station, which change the tuning of the transmitting and receiving apparatus from time to time..."

Two signals are sent, labelled L and R and controlling the left and right rudders of the torpedo. L is indicated by sending a 100 Hz signal over a carrier, R by 500 Hz. Remotely controlled torpedoes had been used before the 1940s, but were often jammed by the target because the control frequency was relatively easy to detect. The innovation in this patent is the use of perforated rolls of paper to modulate the frequency rapidly enough that the enemy would not be able to predict it, making jamming difficult. The perforated rolls of paper were commonly used in player pianos of the time, requiring no special development.

In the patent application seven rows of perforations were used to control the frequency of the carrier. An eighth row of perforations lights a small lamp at the transmitting station. Three of the seven transmission frequencies were dummies which would not actually be received by the torpedo, while the lamp informed the torpedo operator when the weapon was out of contact. The intent of the dummy frequencies appears to be to mislead the enemy and make it more difficult to determine how the control system worked. Some seemingly valid transmission would not be acted upon by the torpedo, while others would.

Player piano tape
Rows A-G tune the radio to one of 7 frequencies.
Row H controls a lamp for the operator when the dummy frequencies A-C are in use.

For the transmitter and receiver to frequency hop in sync, the tape reels must begin rolling at very close to the same time and the speed of the winding must have a reasonably tight tolerance. Machined springs available in the 1930s were sufficiently precise to maintain this for several minutes, long enough to guide a torpedo to its target.

All in all its a fascinating invention which repurposed existing technology for a new purpose, in fighting the Pacific War. Unfortunately the rest of the story is not a happy one, as the invention was not taken seriously by the War Department. By the time the communication industry reinvented spread spectrum communications in the 1950s, this patent had expired.

In 1997 the EFF recognized Ms Lamarr and Mr Antheil's achievement with a Pioneer award.

Monday, March 22, 2010

Somebody Has To Build Them

Business sign for American Skynet.

Hmm. Thats an interesting name for a company. I wonder what they make there?


Terminator T700.

Thursday, March 18, 2010

My Name Is ...

Please search for your name in a search engine, whichever one you prefer. Put quotes around your name or not, use a nickname or not, whatever you like. Now: are you satisfied with what you see in the results? If someone else searches for your name, what are they likely to think?

You don't get a second chance to make a first impression, even when that impression comes from a web search.

This blog was born two years ago because I wasn't happy with the search results for my name. At that time the top Google results were IEEE presentations from years ago on Resilient Packet Ring and Ethernet OAM. So lets check how its going:

Google Search results getsatisfaction.com #1, linkedin #2, listorious #3

Within the top four is ok. The results on Bing are significantly different, this site appears on page three. Bing appears to not automatically treat the two words as a name, so it finds pages where both words appear separately. Enclosing the two words in quotes to enforce proximity refines the search considerably, to the first page.


 
Claim Your Name Online

In 2010, I think everyone should have a URL to use whenever a personal link is needed. It doesn't have to be a blog: it could be your Facebook or Twitter page, a flickr group you contribute to, LinkedIn, etc, just something to link to whenever the situation arises. This question will come up more and more often as the Internet becomes ever more embedded into our society.

Its worthwhile spending a bit of time on the selection. Once you start building up links to this URL it will be difficult to change them. If you point to a domain you don't own, like twitter/facebook/flickr/etc, you have no control over whether they remain reachable. Those services seem like they will last forever, but so did GeoCities and Compuserve in their day. Personally I recommend using a domain name which you own. You don't have to run your own server, for example Blogger, Posterous and Tumblr are all free hosted platforms which allow you to use your own domain name.

Once you've picked a URL which conveys the positive first impression you want, start using it. twitter, LinkedIn, stackoverflow, and many other services you might already use let you include a link in your profile. Comments on blogs (like this one) let you use a URL for your name. You can also sign up for a Google profile, which can link to all of the services you want and will appear on name-query search pages.


 
Strategy: Search Engine Optimization

Once you decide you want to control the search results for your name, it is helpful to understand the basics of Search Engine Optimization (SEO). There is an astonishing amount of information online about SEO, much of it seemingly written by people who consider it a kind of magic for which they exchange incantations and arcane formulae. Most of it makes no sense to me, but a couple things did sink in:

  1. The number of links pointing to a page matters. Writing an astonishingly useful resource which lots of people will link to is a great strategy. I'm still working on that strategy. It turns out to be difficult. Who knew?
  2. When it comes to keywords, DNS > URL > title > body. That is, a keyword occurring in the site's DNS name is weighted more than a keyword in the URL for a page. The URL is weighted more than the same keyword in the page <title>, which is itself weighted more than the body of the page.

 
Tactics: Kneecap the Competition

We're firmly in the web2.0 era, with numerous social web sites vying for our attention. They are also vying for our identity, encouraging people to sign up using their real name. As these are very popular services, they will have lots of inbound links and quickly achieve a higher search ranking for your name than something you create on your own. This is especially true if you use your full name: your profile will contain your name in the URL, which is weighted more heavily. You'll note that in the search results above, two of the top three hits use my name in the URL while the third has it in the <title>.

Stackoverflow change name from denton-gentry to dgentryThere used to be more, with StackOverflow and friendfeed rounding out the top five. Fortunately the username could be changed on both services, relegating them lower in the list. So if you aren't happy with the top results for your name, its often possible to do something about it. You can change your login to an abbreviation or pseudonym, or delete accounts which you no longer use.


 
Think Long Term

Nudging positive material into the search results for your name is one thing. Its equally important, perhaps even more important, to keep material which reflects poorly on you out of the results. Jeremy Toeman published a good article about things one should never do online. I think the common sense rule is not to publish things you would regret later, not even if you think you understand privacy settings or conceal the URL.

Thats all. Please let me know your experiences with the search results for your name.

Monday, March 15, 2010

Marketing Requirements for Food Engineering

One can only imagine the marketing requirements list for packaged food:


  1. As much sweetener as is physically possible to mix in.
  2. Must not list a sweetener as the first ingredient.

Apples, High Fructose Corn Syrup, Corn Syrup, and Water.

Notice how splitting the sweetener components between both High Fructose Corn Syrup and regular Corn Syrup allows Apples to be the first ingredient. I'm surprised they didn't use additional Apple Juice to further sweeten it.

Wednesday, March 10, 2010

Code Snippet: SIOCGIFCONF

A little while ago in this space we discussed SO_BINDTODEVICE, the socket option to control which physical interface will be used for packet ingress/egress. Recently in the comments of that post a question was posed: if you know the IP address of the interface, how do you programmatically find its name?

If there is a direct way to pass in an IP address and get back the interface name, I don't know it. The mechanism I know of is to retrieve the interface list from the kernel and walk through it until you find the IP address you're looking for. The code snippet below demonstrates the technique: the first use of SIOCGIFCONF determines the amount of memory we need, the second retrieves the interface list.

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <sys/socket.h>
#include <sys/ioctl.h>
#include <linux/if.h>
#include <netinet/in.h>
#include <arpa/inet.h>

int main()
{
   struct ifreq *ifr;
   struct ifconf ifc;
   int s, i;
   int numif;

   // find number of interfaces.
   memset(&ifc, 0, sizeof(ifc));
   ifc.ifc_ifcu.ifcu_req = NULL;
   ifc.ifc_len = 0;

   if ((s = socket(AF_INET, SOCK_DGRAM, 0)) < 0) {
     perror("socket");
     exit(1);
   }

   if (ioctl(s, SIOCGIFCONF, &ifc) < 0) {
     perror("ioctl");
     exit(2);
   }

   if ((ifr = malloc(ifc.ifc_len)) == NULL) {
     perror("malloc");
     exit(3);
   }
   ifc.ifc_ifcu.ifcu_req = ifr;

   if (ioctl(s, SIOCGIFCONF, &ifc) < 0) {
     perror("ioctl2");
     exit(4);
   }
   close(s);

   numif = ifc.ifc_len / sizeof(struct ifreq);
   for (i = 0; i < numif; i++) {
     struct ifreq *r = &ifr[i];
     struct sockaddr_in *sin = (struct sockaddr_in *)&r->ifr_addr;

     printf("%-8s : %s\n", r->ifr_name, inet_ntoa(sin->sin_addr));
   }

   free(ifr);
   exit(0);
}

Updates: Mike Ditto points out that the number of interfaces can change between the first call to SIOCGIFCONF and the second, as some workloads result in frequent netdev creation. He advises "ifc.ifc_len = ifc.ifc_len * 2;" before calling malloc. Michael Reed notes that unistd.h is required. It worked for me without it, but only because one of the other includes was pulling it in.

Monday, March 8, 2010

Friday, March 5, 2010

Two Bots Enter, One Bot Leaves

Let me share part of an email I recently received:

I just discovered Coding Relic, which is a really great resource. I really enjoyed your article from last November on Foursquare.
 
<Redacted> is a client of our communications firm, and the topics discussed on your blog are well-matched with the themes of ...

Cylon CenturionEtc, etc. The rest of the email suggested that I write a post about the client. Unfortunately the truth is he did not just discover this blog, in reality he has no idea of its existence. He supplied a list of keywords to an automated tool which searches Technorati/IceRocket/BlogSearch/etc to send an email to those with contact information available. One of the keywords was "foursquare," and it found the one solitary article I wrote which matched.

This email was generated by a bot. It is given an initial direction, which it uses to autonomously pursue a goal. The goal in this case is to generate blog posts about the client. I've chosen not to write a blog post about the client as it is not related to the topics I'm interested in. Instead, I'm going to write about the bot.


 
Warning: Sports Analogies Approaching

This bot plays offense, in that it takes action to achieve a goal for the person who unleashed it upon the net. There are many similar bots which play on offense:

  • Email spam bots should top the list: harvest addresses, send spam email, repeat.
  • Twitter bots will follow, wait a few days for reciprocation, then unfollow in an attempt to build a large follower count. If you do follow back they generate a generic "Welcome and thanks for following my tweets!" message.
  • Hype generation bots, which retweet anything containing a keyword.
  • Google Alerts, which inform you when a site matching specified keywords is crawled by Googlebot.

We're in the middle of an arms race between the bots on offense versus the bots playing defense. Some examples of bots playing defense:

We're still in the early days of networked bots vying for and against our interests. The attacking bots so far have been mostly unfocussed, broadcasted out widely in hopes of catching someone. We're just starting to see the rise of targeted malware, coded with information about the specific person(s) it is trying to reach. The offense is learning tactics and strategy. The defense needs to step up its game. Defensive bots now are specific to particular communications media, monitoring twitter or email or telephone. We're moving to a world where your online presence actively monitors communications channels you use, actively defending your interests.


 
Playing Offense

There is another good bit of advice from the world of sports: You can't win by playing defense. Many useful tools to actively monitor things you are interested in are free, and easily available. Anyone can use them, not just marketing and PR firms. One which I use is Google Alerts, a service by Google to monitor the web index for search terms you specify and email any new results. You can choose to be notified immediately, or to batch results together for a full day or week. You can also choose whether to monitor the entire index, or only specific venues like videos or blog search.

I use Google Alerts to keep track of my online presence, I want to know what the search results for my name will be. "You never get a second chance to make a first impression" is as true now as it ever was, but now the first impression is often formed via a web search for someone's name.


Google Alerts

Monday, March 1, 2010

Brand Control Rage

Ad for Windows Phone 7 podcast.

I wonder if some brand manager at Microsoft is lying awake at night, fuming:

It's Windows Phone 7 Series. The "Series" evokes future promise, the outline of an evolutionary path to reassure the possessor that their investment in Microsoft technology will accrue benefits for years to come. Calling it "Windows Phone 7" cheapens it into just another product, flaring oh-so-briefly briefly before it is trodden into the mud by Windows Phone 8. Curse you, Mary-Jo! Curse you!

Windows Phone 7 Series