We’re jump-starting 2015 with a fresh series of Report In! dev interviews. Kicking off is Mastering Lead Jan Kyncl – a man who often operates from the shadows, but fulfills one of the most critical tasks in Arma 3’s development!
We're often asked to let some of our people talk about what it's like to develop a game at BI. 'Report In!' gives you a more personal perspective on our team, a more detailed look at the way we go about our work, and fresh information about our ongoing projects.
Introduction
Please tell the people a little about yourself. What's your role? How long have you been with Bohemia Interactive? Which projects have you contributed to and what is your favorite BI game or mission?
Hello, my name is Jan Kyncl (nickname Hozki). I joined Bohemia Interactive at the beginning of 2010, which means I've worked here for more than five years now. My first task was to help make configuration files for Arma 2: Operation Arrowhead vehicles, weapons, etcetera. As time passed, my position shifted more and more towards managing the game data. In short, that involves handling 'release candidates', creating patches, making sure that all people working on Arma have the latest up-to-date internal builds where they can commit and test their changes.
You can find my name in the credits of Arma 2: Operation Arrowhead, Arma 2: PMC, Arma 2: ACR, Take On Helicopters, and Arma 3. The latter - Arma 3 - is also my favorite BI game, which is probably not that surprising after all the time we've spent together!
And can you give us a random fact about yourself?
I've studied Tai Chi for one year.
Master of Data
Let’s immediately go into a bit more detail. Can you explain what ‘mastering’ means and why it’s important? Can you give some concrete example(s)?
Well, for me, mastering is a somewhat abstract term which we use in connection with the creating, processing and distributing of data. It's something that reaches pretty much all departments and involves everyone in the development team.
However, to perhaps give a bit more of a concrete example of what we mean by mastering, let's imagine we have a brand new weapon which we want to include in the game. First, we must convert the weapon from a 'human readable' format to one which is optimized and used by the game itself. We call this process binarization or packing. Once the weapon is 'packed', we distribute it to the internal build of the game, where it can be tested, reviewed, and so on. When the weapon is completely finalized, we consider it ready for release and add it to the public version of the game. If then later we find out that there are some bugs related to the weapon, it will be time for an update. And I guess you already know where this all leads to. Indeed, on the technical level, all of these processes and pipelines are handled by the Mastering department.
So what is involved in the internal data management for Arma 3?
Let me begin by explaining that we have two main offices working on Arma 3, one is located in Prague (Mníšek pod Brdy to be specific) and the other in Brno. Therefore, one of the core tasks for the Mastering department is to ensure that the central copy (or build) of the game is always the same for both offices. The second step then is to distribute this central copy to the individual people in the given office.
In the ideal world, regardless of the multiple office situation, all of the changes made to the game are instantly reflected on the computer of every individual person in our team. This is, however, technically not possible as it simply takes some time before a certain change is 'packed' into the game. It might also happen that there are several data packing requests, which means there will be a packing queue. And then the process of synchronizing the newly 'packed' data between offices and to the individual PCs also takes some time.
Besides handling the flow of data internally, you’re also responsible for bringing the correct build of Arma 3 to Steam - effectively handing off the latest version of our game to players. What can you tell us about this process?
It's a longer term process, which usually starts a few weeks before the official release date. The first step is to make a release candidate, which is tested internally, and there's still relatively lots of space to make some adjustments and fixes. After that, we put the candidate to the so called 'rc' branch (rc is short for release candidate) on Steam. There, all people who own Arma 3, and are willing to participate in testing, are free to do so. Changes in this build are less frequent, however, as there is more work involved in handling a build on Steam rather than internally. Also, we want players to have the same data, and to have some time to play before already switching to a newer version.
On the release date we make the latest 'rc' branch build public. By this time we also have new lite server data prepared in order to make the transition to the new version as smooth as possible. Other than that, we are also updating 'tools', 'profiling' and 'legacy' branches but I might be going into too much detail.
Sounds like a lot of responsibility rests on the shoulders of the Mastering department. That must also mean there are a lot of moments of stress and relief?
Yes, I would say so. From the mastering point of view, I feel personally responsible for the state of the Steam build and its various branches on the release date. Once everything is public, and no problem occurs, people will experience our game as intended - and only then I can go home with a clear head.
A Day In The Life
Players are also able to sign up for a development build of Arma 3 (often referred to as Dev-Branch). Here people can help test the latest features and content, and this build is updated almost every day. What does that mean for you?
Similar to the aforementioned 'rc' branch, the Arma 3 development branch enables people to help us test the latest changes to Arma 3 - and this branch is updated almost daily. Thus, each day in the morning I start by checking whether there is some obstacle for publishing the new build to the development branch. Such an obstacle might be a failed auto-test for the .exe, or some fix we want to include in the build which is not yet finished. If there aren't any problems, I can start the process of preparing the data for the development branch. Briefly put, this process consists of including all of the new data, creating a basis for the changelog, and running some automated tests to be sure that we haven't accidentally deleted or included some files, and that all signatures of the data are correct. Another example might be including the right language translations for different regions. Once all of this is done, the build is ready to go to the development branch and then it is just a matter of Internet speed before the build is updated and available on Steam.
So what does a regular day for you look like? For example, with which people do you co-operate the most?
Aside from publishing various builds of Arma 3 to Steam, I maintain the current mastering pipelines. In some cases, it's only a matter of making a few adjustments to some settings file, and in other cases, I have to replace complete parts of the old and/or unsuitable mastering pipeline.
Generally, I'd say I'm co-operating most frequently with the Encoding department - which is responsible for the content of individual addons we use in Arma 3 - and the Programming department, which is responsible for building .exe's which then should be distributed and published. Other than this, every person in the company is free to contact me in case any problem which might be connected to mastering occurs.
What, would you say, are some of the key challenges when it comes to your work?
The first challenge which comes to mind is that I always have to be extremely cautious when publishing to Steam. Therefore, I'm double-checking all data before making it live, and asking other colleagues for the changes they've made if I'm in doubt. Better to be safe than sorry here.
Another one might be the ability to get oriented in the code of someone else as some of the tools used by the Mastering department were developed a long time ago, and you might end up reading an unreadable code which has been swelling over time. I'm working on improving this, however these are things that take some time.
And with something as delicate as your work, has anything simply gone badly?
Not when it comes to publishing data to Steam I hope. Another story is the handling of data internally, where unfortunately it's not an uncommon situation when some person or the whole office are using different data. There might be several, sometimes even unavoidable reasons for this, like an electricity breakdown, issues with the Internet, a stuck server - but let's face it, sometimes it's also just a human-based bug.
From the mastering perspective, what have been some of the most significant improvements to the Steam platform? And what can still be improved?
I mainly recall a problem with data verification in the Steam client. There is an option on Steam called 'verify integrity of game cache...' which should check your local data and fix them if necessary. However, this feature doesn't always work flawlessly and sometimes it might happen that players don't have the right data, even when they are told so. Consequently this can lead to nasty game issues, such as system crashes or the game might simply not work anymore.
Last but not least, you’re currently looking for a Mastering Developer to join the Arma 3 dev-team. What can you tell us about this role, and what should people do to apply?
I hope that by now people have at least some idea of what the Mastering department is responsible for. The role itself will be to some degree based on what the potential candidate wants to work on, but the basic requirements and details for the open position are available on the 'careers' page of Bohemia Interactive. There you can also apply for the job directly, plus we'd like to ask potential candidates to send us some source code of a program they've made.
General Questions
And after a long day of making games, what do you like to do to relax and unwind?
I like to just come home, crash on the sofa with my girlfriend and a glass of wine, and watch some TV series. On weekends, a good way to relax for me is to stay in a cottage in the countryside, without a PC or television, and only one old radio. The calming change can be a good way to recharge.
What advice would you give to who aspire a job like yours?
Be thorough, stay calm, and never be afraid to ask how things work. Also, remember to take some time to think before you start working on the implementation.
A wrathful king has started a burning crusade against computer games, unleashing a Blizzard of Warlords to hunt down all remaining gamers. On the brink of a Cataclysmic World War, and seeking refuge in the mists of Bohemia, what game do you take with you?
World of Warcraft. Even though I've not been playing it much in the last few years (my account is already frozen), I played this for hours and hours before, enjoying its unrepeatable atmosphere.