Simulated Architecture #4: The right tools
Most of my effort in the last while on my Simulated Architecture project has been tool design. I think it's an important part of the project, so I might discuss it for a bit in this instalment.
Having the right tools for the job is essential. Everyone knows this. But given a project, can you choose the right tools? Do you even know what they are? Before I plunge into my project, I've decided to take a step back and evaluate my tool set.
The first important tool is the Integrated Development Environment (IDE) and the associated compiler. Time spent formatting code, writing command lines or building makefiles is wasted time. One of the golden philosophies for this project (and every project) is that if it is tedious or repetitive, then automate it. Get the computer to do it. If the computer can't do it, find a way for it to do so. Putting code on the electronic page and telling a compiler to have at it is easy. I shouldn't have to worry about compiler settings unless I really want to. I shouldn't have to remember the exact spelling and case of a function defined elsewhere. The computer should know all this. Choosing a good IDE gets you most of the way there. I laboured with XEmacs for a while on other projects and had to abandon it. There are more modern, more comprehensive tools out there. I eventually settled on Eclipse. It does all the cool things you want it to: source code highlighting and formatting, auto-complete, auto-compile, function call hierarchies... It's pretty fast and unobtrusive. This is important. You don't want a tool to get in the way of your work. When I had a stab at a modified TADS 3 mode for XEmacs, I spent a fair amount of time battling Lisp and the peculiarities of my mode and XEmacs itself. For what? Nothing. Your tools should augment and enable you. You shouldn't have to augment or enable your computer.
The next important thing is a way to capture knowledge about the project. Commenting your code is only a small part of this. And no matter how wonderful your IDE is at code beautifying, source code is always hard to read to get an idea of the components. Source code is a document of computational process. Literate programming is probably the pinnacle of source code - source that is as readable as any technical document. However, I think for a large project you need to be able to pull back and see the code as objects and relations. This requires detaching yourself from the process of what's happening, and hence, relying less on the source code. The low-level solution I use here is Doxygen. By adding a little bit of metadata and markup to my source code's comments, I can embed concepts inside the code and have Doyxgen take care of showing me relations between code objects. For example, I have an Application object. It controls all the major subsystems in the game: sound, input, graphics, resource management and AI. This is the concept that the Application object has to embody. Naturally, I define member variables like:
class Application {
private:
GraphicsManager mGraphics;
SoundManager mSound;
...
}
Doxygen is smart enough to know about the GraphicsManager and SoundManager classes, and provides hyperlinks to these classes. This documents the implicit relationship between the Application and the various managers. When I'm writing code for the Application, I can browse the relationships between the Application and the managers and get the concepts working correctly, quite distinct from getting the code accurate. It's a fairly simple idea, but it's helpful and effective. It's a step away from the code (unlike elaborate source comments or literate programming) and works best in object-oriented situations.
But still, this documents the code, not the project. They always say: code in the language of the problem, not the language of the machine. This is ideal, but you always need to make compromises and wrangle the computer because it is a pretty stubborn beast. Furthermore, certain subtleties or overarching concepts might be beyond the scope of code comments. People in the business of writing software for a living write design documents and specs. My project is more fluid, more amateur. I don't need to answer to a boss or a deadline. I don't need to worry about the end user so much. My designs can fit this casualness. To this end, I've created a local Wiki. I've downloaded the MediaWiki code (yep, the same that runs Wikipedia). It's easy to install and get running. From there you can design the wiki to capture all your ideas, plans, subsystem documentation and results. It's easy to edit and all you need is a browser, so you needn't worry about Yet Another Piece of Software.
Organizing the wiki takes care. I previously held my Simulated Architecture plans in my main project wiki (which stores all my ideas for stories, games, mathematics and miscellaneous projects). There were two problems with this:
I have to be honest - this wasn't my first idea. My first idea was inspired by Tarn Adams' development page for Dwarf Fortress. He has a whole bunch of enumerated goals, organized into separate webpages. They've got cross-references, so in theory you can tell what needs to be worked on first to achieve a particular set of goals. I emailed him about the site. He's got a bunch of tools to help him with all this (some that store information he doesn't share with the public). Inspired by this, I devoted a weekend to writing a PHP backend that recorded my development tasks in a MySQL database. I even learned how Ajax worked so I could add tasks and not have to refresh pages or anything like that. In the end, I gave it up. It was a lot of work and it didn't look pretty. If I wanted to add something fairly simple, it'd take a lot of modifications. After a while, I realized that I was duplicating a lot of work that a wiki gives you for free (and they do it so much better). So I wrote that down as a learning experience and embraced the wiki.
This is where I'm at, at the moment. I'm starting to flesh out the wiki, storing all my ideas. I've got a template setup for development tasks so if I have the idea to add, say, Shader Model 3.0 support, I just smash out a page and it's all appropriately categorized and linked to other relevant tasks. When the knowledge base becomes very large, I won't have a good birds'-eye view of it all, like which tasks are development bottleneck. Some extra coding in PHP might alleviate this.
In any case, if I want to show people this knowledge base, I can upload the wiki to my website. I don't think I'd want to keep the wiki concept in that case - other people don't have a need to be editing my development plans. I have absolutely no plans to share this project with anyone else (so open source is right out). This may change, so having this sort of potentially collaborative development backend is a good thing.
What else do I need? The next major tools encompass most of the project: I need a tool to create the city, and a tool to view the city. In many games, the tool is one and the same. I think separating them is important in my game as the creation is a hell of a lot more involved than the viewing. But the viewing has its own complications. I certainly can't keep an entire 3d city in memory at once. Memory management for viewing the city is quite different to creating it, and I think there's a lot less coding hassle if I break the two up. Maybe some time in the future it'd be neat to walk around a pre-built city and point to a building and command the computer to fix a bug or just generate something different. At this time, I need to be thinking about creating content or showing it. Not both at once.
Having lobotomised sections of code as separate tools can be useful. For example, having texture generation code in its own program would let me debug texture generation before unleashing it on 3d models. There will probably be things that I can't get the computer to create, so I'll need a modelling/importing tool.
I'll need profilers to gauge general performance and pin-point performance troublemakers. You can profile your C++ code and GPU code quite easily. Once the AI and sound is up and running, getting a grip on its performance would be key - AI especially as it's easy to bring a weighty solution to something that needs to be lightweight.
The last tool is the most important one of the lot: motivation and focus. This is not so easily downloaded and utilised :) But I'm working on it.
Having the right tools for the job is essential. Everyone knows this. But given a project, can you choose the right tools? Do you even know what they are? Before I plunge into my project, I've decided to take a step back and evaluate my tool set.
The first important tool is the Integrated Development Environment (IDE) and the associated compiler. Time spent formatting code, writing command lines or building makefiles is wasted time. One of the golden philosophies for this project (and every project) is that if it is tedious or repetitive, then automate it. Get the computer to do it. If the computer can't do it, find a way for it to do so. Putting code on the electronic page and telling a compiler to have at it is easy. I shouldn't have to worry about compiler settings unless I really want to. I shouldn't have to remember the exact spelling and case of a function defined elsewhere. The computer should know all this. Choosing a good IDE gets you most of the way there. I laboured with XEmacs for a while on other projects and had to abandon it. There are more modern, more comprehensive tools out there. I eventually settled on Eclipse. It does all the cool things you want it to: source code highlighting and formatting, auto-complete, auto-compile, function call hierarchies... It's pretty fast and unobtrusive. This is important. You don't want a tool to get in the way of your work. When I had a stab at a modified TADS 3 mode for XEmacs, I spent a fair amount of time battling Lisp and the peculiarities of my mode and XEmacs itself. For what? Nothing. Your tools should augment and enable you. You shouldn't have to augment or enable your computer.
The next important thing is a way to capture knowledge about the project. Commenting your code is only a small part of this. And no matter how wonderful your IDE is at code beautifying, source code is always hard to read to get an idea of the components. Source code is a document of computational process. Literate programming is probably the pinnacle of source code - source that is as readable as any technical document. However, I think for a large project you need to be able to pull back and see the code as objects and relations. This requires detaching yourself from the process of what's happening, and hence, relying less on the source code. The low-level solution I use here is Doxygen. By adding a little bit of metadata and markup to my source code's comments, I can embed concepts inside the code and have Doyxgen take care of showing me relations between code objects. For example, I have an Application object. It controls all the major subsystems in the game: sound, input, graphics, resource management and AI. This is the concept that the Application object has to embody. Naturally, I define member variables like:
class Application {
private:
GraphicsManager mGraphics;
SoundManager mSound;
...
}
Doxygen is smart enough to know about the GraphicsManager and SoundManager classes, and provides hyperlinks to these classes. This documents the implicit relationship between the Application and the various managers. When I'm writing code for the Application, I can browse the relationships between the Application and the managers and get the concepts working correctly, quite distinct from getting the code accurate. It's a fairly simple idea, but it's helpful and effective. It's a step away from the code (unlike elaborate source comments or literate programming) and works best in object-oriented situations.
But still, this documents the code, not the project. They always say: code in the language of the problem, not the language of the machine. This is ideal, but you always need to make compromises and wrangle the computer because it is a pretty stubborn beast. Furthermore, certain subtleties or overarching concepts might be beyond the scope of code comments. People in the business of writing software for a living write design documents and specs. My project is more fluid, more amateur. I don't need to answer to a boss or a deadline. I don't need to worry about the end user so much. My designs can fit this casualness. To this end, I've created a local Wiki. I've downloaded the MediaWiki code (yep, the same that runs Wikipedia). It's easy to install and get running. From there you can design the wiki to capture all your ideas, plans, subsystem documentation and results. It's easy to edit and all you need is a browser, so you needn't worry about Yet Another Piece of Software.
Organizing the wiki takes care. I previously held my Simulated Architecture plans in my main project wiki (which stores all my ideas for stories, games, mathematics and miscellaneous projects). There were two problems with this:
- If I wanted to open up the Simulated Architecture stuff to people, I'd have to open the whole wiki up, which I wouldn't want to do.
- The structure of the Simulated Architecture pages had to fit the mould of the whole wiki. Adding categories required prepending a title to each, to make sure it was specific to that project. This was a pain.
I have to be honest - this wasn't my first idea. My first idea was inspired by Tarn Adams' development page for Dwarf Fortress. He has a whole bunch of enumerated goals, organized into separate webpages. They've got cross-references, so in theory you can tell what needs to be worked on first to achieve a particular set of goals. I emailed him about the site. He's got a bunch of tools to help him with all this (some that store information he doesn't share with the public). Inspired by this, I devoted a weekend to writing a PHP backend that recorded my development tasks in a MySQL database. I even learned how Ajax worked so I could add tasks and not have to refresh pages or anything like that. In the end, I gave it up. It was a lot of work and it didn't look pretty. If I wanted to add something fairly simple, it'd take a lot of modifications. After a while, I realized that I was duplicating a lot of work that a wiki gives you for free (and they do it so much better). So I wrote that down as a learning experience and embraced the wiki.
This is where I'm at, at the moment. I'm starting to flesh out the wiki, storing all my ideas. I've got a template setup for development tasks so if I have the idea to add, say, Shader Model 3.0 support, I just smash out a page and it's all appropriately categorized and linked to other relevant tasks. When the knowledge base becomes very large, I won't have a good birds'-eye view of it all, like which tasks are development bottleneck. Some extra coding in PHP might alleviate this.
In any case, if I want to show people this knowledge base, I can upload the wiki to my website. I don't think I'd want to keep the wiki concept in that case - other people don't have a need to be editing my development plans. I have absolutely no plans to share this project with anyone else (so open source is right out). This may change, so having this sort of potentially collaborative development backend is a good thing.
What else do I need? The next major tools encompass most of the project: I need a tool to create the city, and a tool to view the city. In many games, the tool is one and the same. I think separating them is important in my game as the creation is a hell of a lot more involved than the viewing. But the viewing has its own complications. I certainly can't keep an entire 3d city in memory at once. Memory management for viewing the city is quite different to creating it, and I think there's a lot less coding hassle if I break the two up. Maybe some time in the future it'd be neat to walk around a pre-built city and point to a building and command the computer to fix a bug or just generate something different. At this time, I need to be thinking about creating content or showing it. Not both at once.
Having lobotomised sections of code as separate tools can be useful. For example, having texture generation code in its own program would let me debug texture generation before unleashing it on 3d models. There will probably be things that I can't get the computer to create, so I'll need a modelling/importing tool.
I'll need profilers to gauge general performance and pin-point performance troublemakers. You can profile your C++ code and GPU code quite easily. Once the AI and sound is up and running, getting a grip on its performance would be key - AI especially as it's easy to bring a weighty solution to something that needs to be lightweight.
The last tool is the most important one of the lot: motivation and focus. This is not so easily downloaded and utilised :) But I'm working on it.