“Ultimately what you want to do is go to a sharded architecture,” said Gorodetzky. “The first 1,000 users go on this set of servers, the next thousand on this set. You’ll find that’s an appropriate way to go, but right now we haven’t done that yet.

It’s also nice to know that they’re at 26 million unique visitors per month without it.

Here is a summary of the tips provided by Digg’s Sys Admin Ron Gorodetzky:

1. Employ master/slave database replication
2. Employ caching to relieve pressure on the database.
3. Scale out your web servers.
4. For horizontal scalability use MogileFS rather than NFS.
5. Experiment with new technologies - They’re experimenting with Puppet for server allocation.
6. Digg uses common tools such as Nagios, and a souped-up version of the Multi-Router Traffic Grapher.

Read the full post here: Digging His Way to Web Success

Related Links:

• Flickr’s Architecture - http://highscalability.com/flickr-architecture
• LiveJournal’s Architecture - http://highscalability.com/livejournal-architecture
• Twitter’s Architecture - http://highscalability.com/scaling-twitter-making-twitter-10000-percent-faster
• Amazon’s Architecture - http://highscalability.com/amazon-architecture
• YouTube’s Architecture - http://highscalability.com/youtube-architecture
• Highscalability.com shard posts - http://highscalability.com/tags/shard

One of the downsides of multi-language development is reusing code across languages. Facebook created Thrift to tie together the Babeling Tower of all their different implementation languages. Thrift is a software framework for scalable cross-language services development. It combines a powerful software stack with a code generation engine to build services that work efficiently and seamlessly between C , Java, Python, PHP, and Ruby. Another approach might be to cross language barriers using REST based services.

This is very cool! I love the idea of being able to marry the strengths of multiple languages. I’ll be taking a closer look at this over the coming months.

Here’s a list of tips:

1. Make synchronized blocks as short as possible
2. Reduce lock granularity
3. Avoid lock on static methods
4. Use lock free data structure in Java SE 5.0
5. Be ware of race conditions: HashMap bug / consider using ConcurrentHashMap
6. Use Non-Blocking I/O for high concurrency (Tomcat uses Blocking I/O, Glassfish uses Non-Blocking)
7. Parallelize single-thread tasks if possible by seeking independence in order of operations. JOMP or Parallel Java may be used to simplify this task.
8. To scale up in memory use multiple JVMs and limit the memory allocated to each to about 3GB or less.
9. Database - pay attention to isolation levels. keep stored procedures to a minimum.
10. Be ware of operating system and disk I/O limitations such as have a large number of files in one directory.
11. Synchronous logging

One of the biggest mistakeproofing challenge is in creating good APIs whether for internal use or external use. We may not even be thinking we’re creating APIs, but any time you create a class or method that another developer has to use it’s an API.

Here are some examples that come to mind:

1. getTaxRate(String country, String state) - it’s possible to pass these parameters in the wrong order.
2. Connection dbConnection = new Connection(String longComplicatedConnectionString) - I hate this one, I often have to look at the docs to figure out how to construct the connection string.
3. callRemoteMethod(String nameOfMethod, Oject[] params, long timeoutperiod) - many problems with this… but I’m thinking of “what if I use a timeout period that’s too short for a particular call?”
4. Any time I have to call a series of methods in the *right* order.

There are endless examples of APIs that are easy to misuse. Spend some time and you’ll easily be able to come up with a few.

The next time you’re writing software on a team remember that you’re probably creating an API of some kind that has the potential for accidental misuse. Think about how the API could be misused and try to find a way to structure it so that misuse not possible or extremely difficult.

Here is the list of best practices he recommends:

1. Make fewer HTTP requests
2. Use a CDN
3. Add an Expires header
4. Gzip components
5. Put stylesheets at the top
6. Put scripts at the bottom
7. Avoid CSS expressions
8. Make JS and CSS external
9. Reduce DNS lookups
10. Minify JS
11. Avoid redirects
12. Remove duplicate scripts
13. Configure ETags
14. Make AJAX cacheable

Watch this Video of Steve’s talk at the recent Google I/O conference:

Even Faster Web Sites (Google I/O Session Videos and Slides)

http://www.scribd.com/doc/40437/Commodore-64-Users-Guide

The news today that Sun hired two key Python programmers makes me hopeful that they’re thinking what I’m thinking.

Ultimately I suggested he take a look at some of the material at xunitpatterns.com, a site developed by Gerard Meszaros while he was authoring the book xUnit Test Patterns. I was fortunate enough to be mentored by Gerard on several projects I worked on in Calgary, and have a tremendous amount of respect for him and his knowledge when it comes to testing software.

After looking at xunitpatterns.com Dustin and I realized the real question was what type of Test Double should be used. For the problem at hand, we decided to go with a Configurable Test Double. If you are stuck on a testing problem, or just want to improve your testing skills, I strongly recommend looking at Gerard’s site and picking up his book.

public void printPowers() {
    for (int r = 0; r < 100; r++) {
        int x = Math.pow(r, r);
        System.out.print(x);
        System.out.print(",");
    }
}

In Java - *with class definition and formatting left out

[r**r for r in range(100)]

In Python

Another thing I didn’t mention: The java implementation won’t work because int can’t store 99^99, but Python takes care of handling arbitrarily large numbers for you. It’s great.