Flickr is one of my favorite and a leading photo sharing website. Flickr must be having some amazing challenges when it comes to managing their content. Ever expanding new content, ever increasing new users and constant stream of new features and while it provides excellent performance. How do they do it?
Site: flickr.com
Flickr and PHP (an early document)
Capacity planning for lamp
Federation at Flickr: Doing Billions of Queries a Day by Dathan Pattishall.
Platform
PHP
MySQL
Shards
Memcached for a caching layer.
Squid in reverse-proxy for html and images.
Linux (RedHat)
Smarty for templating
Perl
PEAR for XML and Email parsing
ImageMagick, for image processing
Java, for the node service
Apache
SystemImager for deployment
Ganglia for distributed system monitoring
Subcon stores essential system configuration files in a subversion repository for easy deployment to machines in a cluster.
Cvsup for distributing and updating collections of files across a network.
The Stats
More than 4 billion queries per day.
35M photos in squid cache (total)
2M photos in squid’s RAM
470M photos, 4 or 5 sizes of each
38k req/sec to memcached (12M objects)
2 PB raw storage (consumed about ~1.5TB on Sunday
Over 400,000 photos being added every day
The Architecture
A pretty picture of Flickr's architecture can be found on this slide. A simple depiction is:
-- Pair of ServerIron's
---- Squid Caches
------ Net App's
---- PHP App Servers
------ Storage Manager
------ Master-master shards
------ Dual Tree Central Database
------ Memcached Cluster
------ Big Search Engine
Use dedicated servers for static content.
Talks about how to support Unicode.
Use a share nothing architecture.
Everything (except photos) are stored in the database.
Statelessness means they can bounce people around servers and it's easier to make their APIs.
Use horizontal scaling so they just need to add more machines.
Shards (google it)
Every users reads and writes are kept in one shard. Notion of replication lag is gone.
Average queries per page, are 27-35 SQL statements.
Each Shard holds 400K+ users data.
ibbackup on a cron job, that runs across various shards at different times.
Snapshots are taken every night across the entire cluster of databases.
Photos are stored on the filer. Pointers are stored in the database.
Friday, May 14
Top 3 iPhone word games
Word Scramble 2
By Zynga - This comprehensive free offering gives you the choice of playing solo, play and pass with up to four friends, playing online with random folk and a challenge mode for you to throw the gauntlet out to a buddy who also has the app or plays on Facebook, which is a sweet touch.
Words with Friends Free
This might actually be the best. game. ever. Think Scrabble with the option to play your Twitter and Facebook buddies on your iPhone. If you’re short of social networking pals that like word games then the app will also let you invite someone to play from your address book, and will even randomly match you with another willing player from the iPhoneiverse. Or, if you have actual real life friends who like a bit of word game action you can opt to “Pass and Play” and play with them.
WordJong Lite
This popular app puts a word game twist on Mahjong, with great results. The basic principle is to make the best words possible from the top layer of tiles available to you. The app is a little on the girly side, with pink blossoms filling up the calendar as you complete a game and collecting butterflies is the prize, but if you can get past that, then it’s a simple yet clever time-filler.
By Zynga - This comprehensive free offering gives you the choice of playing solo, play and pass with up to four friends, playing online with random folk and a challenge mode for you to throw the gauntlet out to a buddy who also has the app or plays on Facebook, which is a sweet touch.
Words with Friends Free
This might actually be the best. game. ever. Think Scrabble with the option to play your Twitter and Facebook buddies on your iPhone. If you’re short of social networking pals that like word games then the app will also let you invite someone to play from your address book, and will even randomly match you with another willing player from the iPhoneiverse. Or, if you have actual real life friends who like a bit of word game action you can opt to “Pass and Play” and play with them.
WordJong Lite
This popular app puts a word game twist on Mahjong, with great results. The basic principle is to make the best words possible from the top layer of tiles available to you. The app is a little on the girly side, with pink blossoms filling up the calendar as you complete a game and collecting butterflies is the prize, but if you can get past that, then it’s a simple yet clever time-filler.
Thursday, October 8
Google Trike rides into Singapore
The google tricycle is so coool!! They did google street view for the F1 track. Wondering when is the street view going to be ready!!! The Street View Van was spotted going around Singapore last year. Until now still not up yet. Maybe they are redoing the whole street view with the Google Trike?
The search giant is adding what it calls a "mechanical masterpiece" to gather images at places inaccessible by the car.
According to Google, this is the first time the Trike is used in Southeast Asia (second time in Asia Pacific after Japan) and Singapore will still be the first country to get Street View in SEA. The company has also teamed up with the Singapore Tourism Board to conduct an online poll where Singaporeans can suggest where the Trike should visit to collect the imagery. Ideas can be submitted in five categories: The City, Cultural Areas, Natural Wonders, The River & Quays and Hidden Gems. The poll runs from now till October 12.
Wednesday, October 7
Wake up SID
Being honest, I went to watch the movie with expectations of just being entertained...what i had on mind was a movie which was fun, young and entertaining. Well, i was in for a pleasant surprise...Yes, the movie was fresh, young and entertaining indeed...but what came as a huge bonus was the maturity with which the subject was handled. it turned out to be one rare experience where sensitivity and simplicity meet maturity and sensibility. All the cheers to Mr. Mukherjee for giving it the right treatment. turning a "one line story" into an experience that every one can relate to and take home more than just a few entertaining moments...is a task well acheived by the director. Hard to believe that it was his first directorial exhibition. Screenplay is a winner all the way...the dialogues are straight out of normal daily life. everything ranging from the attitude of today's youth, their relationship with their parents, the bodning between the friends and the development of relationship between ranbir and konkona are well-balanced. The soundtrack is one of the best in recent times. for once we have a soundtrack that doesn't hinder the flow of movie...rather it plays a very very important role in enhancing the impact.
it was such a pleasant change to watch the on-screen pairing of Ranbir and Konkona and their relationship. For once we have an onscreen couple which are connected through minds rather than bodies.
Rahul Khanna is wasted once again. Kashmera Shah has nothing to do. Sid's friends are PERFECT. Overall...It is one of the best to hit the screen in last few years. Not to be missed at all.
Subscribe to:
Posts (Atom)
