I was recently confronted with very interesting misconception: that Agile and SCRUM are the same thing. It really got me thinking. First, it means that one must be very careful when using the word “agile” in software development context. Less obvious: SCRUM is de facto standard agile method today, and very often the only one people know or heard about.
Wikipedia lists 10+ different methods falling under “Agile” umbrella. It’s no suprise because agile manifesto is very vague and does not really tell much (if anything) about the how part. Specific methods declared as “agile” are simply inspired by these principles. Agile software development is a way of thinking, or even a philosophy, developed as a reaction to shortcomings of “traditional” methods. While this religious/flamewar flavour may sound dangerous, it’s a fact that SCRUM (as an example) solves some of these shortcomings in a very neat and precise way.

But is Agile any better?

Short answer: Typically, any method is better than no method at all (sometimes called DWYW – Do Whatever You Want method).
Even shorter answer: it depends.
Now the trick question: What is your definition of “better”?
Then comes even tougher part. Here are some question about the company, team, project that you might want to think about for a while:

• What is your organization strategy?
• What is your current software development process, if any?
• What are the deficiencies of the existing process that make you think about adopting an agile method?
• What is the problem you are trying to solve?
• What is the problem domain?
• Do you have domain experts on site? Is the customer willing to provide them? On what terms?
• Is the problem repeatable?
• Who is your customer? Is it internal or external customer? Is the customer willing to work in a specific way?
• Do you have a fixed budget and/or amount of time available?
• Can you accept failure? to put in other words: what is your risk aversion?
• What do you consider be a failure?
• How well are the requirements defined?
• How likely are the requirements to change, and by how much?
• Are there any other cosiderations (i.e. security, outsourcing, legal) that are relevant to an organization or specific project?

I could go on and on. If you noticed that these questions are hardly technical, good for you The most important takeaway here is very simple: software development does not ever exist in a vacuum!
The primary goal of any development team is to meet the needs of customers (and, by proxy, the business as a whole) and do it in the most efficient way. This is the only metric that really matters, because ultimately companies exist to make money, and without customers they cannot. Everything else is secondary. If waterfall development is the answer, by all means go for it. If not, go with something else. It’s a business, not religion!

The perfect software development process

I do not believe there is “one method to rule them all”, or that there will ever be one. Having said that, assuming specific company, market, team, project, etc etc, there definitely exists some perfect process for that composition. At the same time, you probably will not be able to identify and introduce it for very simple reason: by the time you do, the world will change. So, instead of thinking about what’s better in principle start thinking about what’s better right here, right now. Start thinking about agile process development.

The next generation of consoles will most likely come by the end of year 2013. One intersting question arise: how it should look like? Disclaimer: by “should” I mean “actually makes business sense for both platform holders and developers” and not “what hardcore gamers would like to see”. These meanings are at quite a big conflict, as we shall see later

In order to think about next gen we have to take deeper look at the general trends in the entertainment industry. Some of these trends were not significant at X360/PS3 launch, some did not even exist at the time. The world has radically changed in the past few years, invalidating many assumptions entire console industry was built on.
As a reminder, the classic console business model is based on selling hardware at a loss in order to build userbase and later getting the money back in the form of licensing fees (which implies absolute control over platform). Traditionally, consoles were very powerful at release – comparable to very good gaming PC. Other than cost optimizations and physical layout, during lifetime of a console hardware specifications are fixed. This provided value to customers in two ways: one way was getting very powerful hardware at a discounted price, and second was getting stable hardware configuration that is guaranteed to be compatible with every game released for the system. Of course, console gamers eventually end up paying more than PC gamers but because it is spread over long long time, it’s something people generally tend to overlook.
It’s all great – if it works. However, if the console doesn’t get enough traction it is a financial disaster. Developing custom hardware and operating system is insanely costly and time consuming. We’re not only talking custom boards and software here – we’re talking CPU/GPU design! At launch, instead of starting to make money on the console, even more money is spent. It’s a huge risk to take.
Subsidizing the hardware is something that was already put into doubt at the very beginning of the current console cycle by Nintendo. Wii was never sold at a loss, allowing the profit to be made on both games and hardware itself. It is reasonable to assume that designing the Wii was a lot cheaper than designing PS3/X360 – essentially is was slightly beefed up Gamecube with a new controller. Obviously, the break even point for Nintendo was much lower than for Sony/Microsoft, something I am sure both companies noticed.
What everyone noticed is that there’s huge group of people who didn’t mind worse graphics quality, and didn’t care about HD that much. I am not trying to convince anyone here that better graphics and all other stuff brought by high performance hardware do not matter. They do. But with every hardware generation the group of people who care is actually smaller and smaller.
Now some personal observation. While I do not play games on a PC that much, I managed to notice one thing related to console cycles. Every time game console is released PC players say “wow!” (deep inside, of course, no PC gamer will openly admit that ). Shortly (a year or so later) PC hardware and games are roughly the same quality. Fast forward two years and PC games are vastly superior (this is the moment when PC gamers proudly bash console gamers in public ). Finally, when new console generation arrives, everyone says “it’s about time”. I don’t see this cycle happening right now … sure, there are some DX11 titles that look very good (and look at tech demos from companies like Epic and Crytek!) but I don’t see people agonizing over the need for new GPU, or even complete system. Also, I don’t see people getting excited about new chips from AMD/Nvidia or new Intel CPUs. Hardcore gamers might still be in denial, but I’ll say it anyway: graphics quality is at the stage where providing even small gains in user experience requires expotential growth in computing power. In order to make people say “wow!” again next gen systems would need a lot.
How much? For starters, we need to double GPU power just to port existing content to 1080p. I’d say that for a meaningful boost in quality, we’d also need 4x programmable shading performance on top of that – so 8x the GPU performance as a bare minimum. It’s safe to assume we’d also need a lot more memory – just boosting the texture resolution 2x requires 4x the memory. But we’d want to have HDR rendering pipeline. Ooops, that’s bigger framebuffer, and probably some HDR textures. Not to mention a lot of modern rendering techniques require floating point buffers and textures anyway. Add more geometry storage, too (even if in the form of displacement maps). So, going “the old way” next gen console would need at least 4GB of total memory and 2TFLOPS of programmable GPU, all that paired with significantly faster CPU. This is doable even today, but even if you ignore costs, there is another catch in here, and that’s … power consumption.
The correct way of thinking about CPU/GPU today is only one: performance/Watt. Fitting above spec within 200W envelope is going to be very challenging, even with 22nm CPU/GPU process. Can Sony/MS take a risk and go with a lot more powerful hardware at the expense of noise and power usage? I don’t think so. But that’s exactly what they would have to do make games significantly better looking.
Now look for a while at the development side of things. Some game budgets are already in line with Hollywood movies. Next gen as described above would require a lot more content, and new content production pipelines. Insane budgets would only get bigger … There’s room for very few blockbusters out there, and even for them the business is getting tougher and tougher. Money and effort is likely to naturally flow in the direction of lower risk and higher ROI, and might mean moving away from traditional consoles. It’s not like developers don’t have a choice.
In fact, right now developers have more choice than ever! First of all, there’s iOS and Android, in both smartphone and tablet form factors. These platforms have relatively low barriers to entry and and are natural target for small/medium development teams. There’s also Steam on PC, along with some smaller online stores. Last but not least, there’s Facebook, along with smaller social networks. Any of these are (from developer point of view) better platforms than XBox or PlayStation.
That’s something significant even for big players in the industry. Simply speaking it’s better to invest in 5-10 small projects than in single big project – it is likely that some of the small projects will perform below expectations, or even fail, but at the same time it is unlikely that the entire investment portfolio will be a loser because the risk will be highly diversified. A portfolio of big projects is more and more like playing poker with your entire bankroll on table … Current state of THQ clearly shows what the effects can be – positive expected value is not enough to gamble with hundreds of milions of dollars (although some investment bankers thought differently ).
Platform without support of developers is a dead platform, and developing for consoles in the classic model is not really attractive any more from business point of view. XBLA and PSN might still be a viable target platform, but as they are it’s harder and harder to justify making any of them the lead platform – what we see more and more is that game gets launched on iOS/Android/Steam/Facebook and then possibly ported to consoles, not vice versa.
Pressure from other platforms is higher than ever, and I believe it is actually underestimated, even by some game industry insiders. Tablets and smartphones are right now at the stage of transitioning from luxury to commodity, and this process will only accelerate. Today iOS/Android devices are quite expensive. Imagine sub-$100 tablet with dual core CPU, 1GB RAM, 720p display, and network connectivity. That will happen a lot sooner than people expect it – there are already some very, very cheap Android tablets. Sure, iPad 4 will be more powerful than this, but it’s the low end where real changes will happen. This is going to totally change the economy of computing because tablet will actually be cheaper than a PC. Think about it: tablets will be used by people who never owned a PC in the first place! This hardware will also make its way to the TV sets eventually … we might even see game console released by Apple (I wrote about the possibility not long ago ago).
Good thing: more people than ever will own some hardware capable of playing games, and the game industry will grow. Bad thing for those who cannot adapt: the composition of gaming market will also change. Developers are quickest to make the shift because of shorter product cycles but console makers also have to adapt to the changing world. Their primary targets should be:

• Lower hardware costs. $600 at launch was shock few years ago, it is impossible now. Console should cost $250, $300 max, and that’s without subsidizing it in any way.
• More flexibility in pricing and business models.
• Lower barriers to entry for developers.
• Focus on building services to which console is only point of access, perhaps just one of many.

So, going back to initial question: how should the next generation look like? Here’s my checklist:

• Off-shelf GPU at DX11/OpenGL 4.x level, with 1 GB of memory, 600-800 GFLOPS, 60-80 GB/s memory bandwidth.
• Off-shelf quad core Intel CPU with HT, clocked at 2-2.5 GHz, or similar AMD CPU. Definitely x86, for reasons I’ll explain below.
• 2 GB of DDR3 system memory (4GB might be reasonable tradeoff)
• No optical disk drive. Physical distribution would still be supported but using proprietary memory cards (the same way PS Vita does it) – it greatly reduces costs, keeps power usage smaller, and has advantage of fast random access (possibly reducing need of installing a lot of data locally).
• Internal 120 GB HDD (possibly slightly bigger because of low marginal cost of additional capacity)
• Kinect/Move in every box and/or some sort of touch/tablet controller (Wii U, anyone? ).
• HDMI output, Wi-Fi 802.11n, Gigabit Ethernet, Bluetooth, possibly USB ports.
• AppStore business model for electronic distribution – simple and well defined approval process, developer sets the price, 30% revenue cut for platform owner. ESRB rating optional. Mature content allowed (but filtered). Freemium and in-game purchases allowed. Non-game applications allowed.
• Development model: standard APIs and tools only, no direct or exclusive hardware access. Essentially, PC-like model but with applications digitally signed and isolated from each other.
• 1080p used in the UI and suggested for games, but not required.
• One CPU core dedicated to running background applications, which can also be written by third party developers. Messengers, streaming audio/video services, social media apps … the only rule: if money is involved, 30% goes to platform holder
• Apple-like hardware cycle – refreshed version every 18-24 months or so, backwards compatible. It’s up to developers to decide if they want to support older hardware. 5-7 year cycle is in my opinion not realistic any more.

Using standard components means lower hardware design costs and later lower manufacturing costs. This device could possibly be sold at initial price of $250 (at the end of 2013) with no loss. Price point is friendly enough to attract casual gamers and casual game developers, especially with the specification above (move/touch control, non-gaming apps, AppStore model, easy development …). At the same time, console like this one would be powerful enough to easily run existing content in 1080p, although without major content/engine upgrades.
If you haven’t yet noticed, what I suggested above is simply a locked down low end PC, in a configuration and form factor that’s suited for gaming. It is an interesting opportunity for Microsoft – they could actually build slightly modified Windows 8 PC and sell it under XBox brand. Will any of that happen? I have no idea, it makes sense to me. We might still see a beast with 3+ TFLOPS GPU + 16-core CPU, that makes neighbor lights go dimmer when turned on. But is anyone actually interested in it? Or in developing games for it? I don’t believe so.

It was quite some time ago when my company started using Fusion-IO (enterprise SSDs) on our database servers. We are happily using them till this day. The great part is that we could scale vertically and leave software layer unchanged. Even better: we still have room for growth, despite getting more users and our application stack getting more and more demanding. We were one of the first to switch to SSDs on server side – so right now it’s interesting that this idea propagated so quickly. We got to the point where people start loudly saying “disk is the new tape”. And it really is.
The gap between CPU speed and external memory has been widening for a long time, and got to the point of being simply ridiculous. It’s worth noting that it’s not only about gap between CPU and disk drive but also about gap between CPU and DRAM (which is slightly less known fact).
For CPU accessing L1 cache takes single digit amount of clock cycles. Subsequent levels of memory hierarchy are slower and slower. Accessing main memory gets us into area of hundereds to even 1000+ CPU cycles (this is strongly architecture dependent). This is very bad and low level programmers have been battling this issue for a long time now. One of approaches is using access patterns that minimize CPU cache misses and using low level prefetch instructions. We are essentally talking about linearizing access to memory that is supposed to be “random access”. If latency is the problem in case of DRAM, how bad it is in case of disk drives?
Very, very bad. Let’s start with simple analogy. I need something and:

• it happens to be in my room (that’s my L1 cache) – it takes a few seconds to get what I need
• ok, it’s not on my desk (I have to hit DRAM) – it takes a few minutes to get what I need, an equivalent to, say, walking to a neighbor and borrowing something
• hmmm, actually my neighbor doesn’t have it and I have to hit the store – how much would it take if the store was the disk?

This is where 99% of people totally fail at estimation saying something like “few hours”. The real answer is: much, much longer.
Let’s assume that DRAM latency is at 100 ns (so about 300 clock cycles of 3GHz CPU), and average disk seek time is 6 ms (a very fast disk!). The difference is 60 000 times! We are not talking about “get into a car and drive to the store”. In our hypothetical scenario we are actually talking about months!
But it’s only part of the story, because it’s not just about latency – it’s also about parallelism. For a hard disk drive latency determines the upper number of random I/O operations per second because disk has to do everything serially. The serial nature of HDDs is the real problem and source of pain as CPUs are becoming more and more parallel, putting more and more processing power into roughly the same space. Because CPU throughput is skyrocketing (high end Xeon E7 is 10 cores/20 threads), we need more and more I/O operations per second. Single hard disk cannot provide these, and will never be able to
In fact, most of server side programming nowadays revolves around optimizing for I/O (in general), not for CPU. This happens at many levels – operating system kernel, file systems, database storage engines, applications … Tons of software engineering work was performed just to hide simple fact that the disks cannot keep up.
Interesting consequence is the proliferation of Hadoop (and other implementations of MapReduce) as an alternative to relational databases in many data processing tasks. Hadoop takes what’s good about disks (cheap bandwidth) and gets rid of the latency problem by streaming all the data, instead of randomly accessing it.
SSDs still have latency of ~0.1 ms – it a lot faster than HDDs, but at the same time a lot slower than DRAM … What’s special about SSDs is not the access time – it’s the parallelism. Enterprise SSDs offer hundreds of thousands of I/O operations per second – the difference here is a lot bigger than the difference in access times. Additionally, while the performance of disk drives is not really improving any more, Flash memory based solutions have huge room for improvement – and their I/O performance is going to improve as fast or even faster than CPU performance. One very simple reason for that is that right now we are still trying to access them through traditional file I/O interfaces that weren’t meant for this kind of usage …
In fact, when we start treating SSDs for what they really are, a memory, and forget about block device+filesystem nonsense, we are going to see massive performance gains. It’s already happening in the form of Auto Commit Memory technology from Fusion-IO. Surely more will follow, because right now interfaces and API layers are huge source of inefficiency. Also, it’s good to know that NAND Flash is not the last word in technologies of non-volatile memory and there are others on the horizon (although I am not aware of any commercial applications right now).
We are heading towards a world where speed gap between HDD and SSD is measured in thousands of times, and getting wider every day. Unfortunately, because it’s a revolutionary shift, the price of Flash memory does not go down as fast as one might want it to. Simply speaking, now everyone wants all of their hot data to be in Flash memory. Massive demand for Flash memory in consumer electronics does not make things any better – Apple is likely sucking in significant chunk of world production
Interesting question is: where are the SSDs in the cloud? Recently, as it turns out, Amazon used a lot of these to launch DynamoDB. Which from my point of view creates interesting problem – because it becomes essentially impossible to build comparable service on “raw” EC2. That means getting locked into AWS. Will Amazon introduce some sort of “high I/O EBS”? I am sure I am not the only one interested in such product.

So here you have it, my new presentation. This time it’s about the benefits of using cloud services for hosting, combined with introduction to Amazon Web Services. I’ll be giving it tomorrow at Ganymede to our developers.

It doesn’t contain anything not known to public (other than the fact that Ganymede is indeed working with AWS – but this is hardly a secret ). I would really love to cover some details of our game server architecture one day – this is where some really cool stuff is

Recently I somehow found enough time to read “OpenGL SuperBible, 5th edition”. It was something done purely for fun, just to see how things are looking today for modern graphics APIs.
Last time I did any serious graphics programming, OpenGL was slowly dying. In fact, my M.Sc. thesis used Direct3D as implementation API. I simply had no choice. There was no standard for anything – every single important feature was exposed through extensions. At the same time, Direct3D 9 was well thought, simple, and forward looking. Microsoft was leading the way, and everyone knew that. As an example: somewhere around 2005, EXT_framebuffer_object appeared, giving GL functionality that was available since Direct3D 7!
Good news: things have changed, and they have changed for better. In fact, todays OpenGL is awesome. And that’s despite Microsoft API advancing two versions forward since then! Extension hell is gone – core OpenGL defines 99.9% of the stuff you need (I think that the only universally supported functionality that is not in the core is anisotropic filtering and S3TC formats). And there is such thing as “core profile”, making OpenGL shader-only API, with all the legacy stuff removed. What’s also great is that GL went the way of D3D and states pretty clear minimum requirements to support any feature: texture units, inputs and outputs of specific shader stages, texture and renderbuffer formats etc. These minimums are set quite high, and generally correspond to what you see in D3D. Now, I am sure thare are still some differences between the two APIs, but I believe they are pretty much irrelevant.
Let’s go back to the book, because by that time you might be wondering “is it any good?”. The reviews on Amazon aren’t really favorable but my personal opinion is that it is a very, very good book to teach the API and get yourself started with modern GPU programming. Author does not waste time describing legacy fixed function functionality, which is a big plus. You might wonder how to achieve that without falling back to specification-like language – in shader-only environment there’s a lot you have to do before anything actually appears on screen: write, compile and link your shaders (even if simplistic ones), upload geometry data to the GPU, activate shaders, set uniform variables, draw the geometry … a lot of things to do, and you can’t really understand it bit by bit. The book solves “chinken and egg” problem in very elegant way of its own utility libraries. Sample code relies on utility functions that are gradually replaced/extended with newly introduced functionality.
OpenGL SuperBible is not about complex rendering algorithms and shader authoring – every sample is meant to illustrate specific API functionality. At the same time samples are not artificial, which is a big plus (i.e. there is clear example of why you might want to to multisample resolve in the shader in case of HDR rendering). Also, this book is not going to teach anyone about inner workings of modern GPUs. I think that clear focus on just teaching the API is the right thing. Considering the target of this book (beginners) it’s also really hard not to forgive that math part isn’t very thorough
For me it was a very good read simply to catch up with things, but also I think this book is very good place to start adventure with graphics programming, and then continue education with more advanced resources. Even then, this book will probably remain a valuable reference that you’ll keep handy