Skip to main content

Case Study: Spotlight

Learn how distributed achitecture transformed the company's business.

Watch video

Incremental changes are the name of the game for Dylan Beattie. As IT manager and system architect for Spotlight, he keeps the company’s casting operations -- data and photos of over 60,000 performers, agencies, and directors -- up-to-date with current technologies.

The core of Spotlight’s business has been managing and delivering photographs since 1927. They went online in 1997 and Dylan joined the company soon afterwards in 2000.

The importance of Spotlight’s website has since grown substantially and spotlight.com is now the primary channel for most of their casting services. As more services were moved online and web traffic increased, Dylan found some of their legacy systems were struggling to cope with the increased load, including the system used for publishing photographs on the web.

Spotlight’s thumbnail server was designed for an earlier era of light web traffic with a central file server and pull scripts. Master images were kept on a central file server with web servers generating –- and caching –- thumbnails on demand. Anecdotal reports of “ the website is running slowly ” grew more common, from weekly to daily, and it became clear there was an underlying problem that additional server capacity could not solve.

Uncovering the problem

Rendering photographs was the root cause of their web slow-down. Analysis of IIS logs, Windows performance counters and website traffic patterns showed that 70% of the web servers’ CPU time was spent generating thumbnail photographs. In the legacy system, every time a user requested a photograph, the web server had to compare the local thumbnail rendition against the full-size image stored on the central file server to ensure the thumbnail was still up to date.

The question for Dylan was how to provide accurate, up-to-date renditions of hundreds of thousands of high-resolution photographs - while minimizing the CPU load on Spotlight’s web servers, and the network traffic between their website and their main file server.

Meeting the technical challenges of the business model

Solving these technical issues required understanding and building around the Spotlight business model. Our photography publishing system is linked to a book publishing system, which is linked to a revenue system, so we actually had quite a lot of challenges, says Dylan.

It was not possible for Spotlight’s team to create a greenfield system from scratch – as he would liked to have done: We couldn't do something sensible, like give a photograph one file name and then if there was a different picture, to give it a different file name, he points out. Instead of using unique URLs, the legacy publishing workflow meant we had to re-use the same filenames and URLs for modified images, which makes HTTP-level caching very difficult to use effectively.

Discovering the potential in distributed architecture

Spotlight came with a mixed IT background. Their core data was mainly stored in Microsoft SQL Server, which influenced the technology stack. Their earliest website was built in classic ASP, and over the years they’d begun adopting ASP.NET and various open-source .NET frameworks, but the underlying architecture was still very monolithic.

We were struggling to scale with the environment we had, remembers Dylan. We ran everything off one database server, one file server and two web servers, and when things started to slow down, we would just buy a bigger server -- until we realized that we’d hit the limit, and they just weren’t selling any servers bigger than the ones we already had!