Engine Tech Update

It’s been a while since our last post, but we’re working towards some big updates to share this fall.  Today, I’ll highlight a few key engine improvements that have helped us develop Thumper.

Visual Scripting

The benefits of a tool that enables non-programmers to hookup effects, animations, sounds, etc. with simple logic are obvious — every major game engine has something like this.  But as a small developer with a custom engine, the cost of building such a complex tool can be hard to justify.  You want to make games, not tools, after all.  We started this project using a simple property-grid based system and custom object editors, but as the game became more complex, it became unsustainable.  Brian was often blocked, waiting for me to create or extend custom objects.  So we finally decided to roll our own visual scripting tool.  Here’s what it looks like.

Visual Script Example

While implementing, I was concerned with performance and this post on the BitSquid engine’s system was a huge help.  By focusing on memory layout up front, we have a scripting tool that is cache-coherent and fast.  Including the UI, it took about a month to implement.  It was worth it.

With this tool, Brian can do much more without my involvement.  It’s easy for me to create custom script nodes in C — that’s a good way to expose more complicated functionality as needed.  I’m tempted to integrate a scripting language (like Lua) as an intermediary between our visual scripts and C, but for Thumper, that’s probably not necessary.

Single-step mode

I added a “single-step mode” where you can pause the game/editor and advance it by a single frame with a key press.  It’s simple, but useful.  Brian often uses this to closely examine animation and effects.  Thumper has a lot of fast animations and quick interactions, so this helps us make sure everything looks and feels completely solid.  While in this mode, the main game loop is not run and the time line is not advanced.  That’s convenient when I’m debugging and trying to reproduce particular conditions or set breakpoints.

Its simple to use, you just press F6 to enter single-step mode.  While in the mode, you press F6 again to advance one frame, and Ctrl+F6 to exit the mode.  Here are a few frames of an animation, each advanced by a single frame.

Single-Step Mode

Entities and Instancing

Thumper has many different instanced gameplay objects, or entities, including rhythmic cues, decorative meshes, effects, enemies, etc.  Initially, we developed entities in an ad hoc way.  Brian would create the raw resources and custom code would allocate memory for each instance and stitch everything together.  It was OK for prototyping, but things got gnarly fast.

So I finally built a real entity instancing system.  Now each entity has its own resource file with a well-defined interface that can be extended with our visual scripting tool.  Draw calls for instances are batched.  Efficient memory allocation for dynamic entity instances was a challenge, but I found the sections on game object memory management in this game engine architecture book useful.

This screenshot from our test mode shows the obvious benefits of an entity instancing system.

Blubber Instancing

Debug Visualizations

I created some high-level visualizations of what’s going on in the game.  Including things like audio playback, animations, tasks, memory usage, etc.  This is another example of something easy and simple that’s indispensable once you have it.  Here’s a visualization of active animations and tasks.  It’s hard to overstate how useful it is for debugging.

Task Visualization

Next Steps

We’ve come a long way, but some big tech challenges are ahead.  Over the next couple of months, I’ll add automatic draw ordering for our renderer, a “shader graph” for creating custom post-processing effects, and a packaging system for our builds, to name a few of the big features.  I’ll post more about our tech developments as they’re implemented.

Is Thumper a “Music Game?”

We’re asked that question often and there’s not a simple answer.  It depends on what one means by “music game.”   The label efficiently conveys some elements of our game, but it’s potentially misleading and limiting too.

Fortunately Dan Soldberg has skillfully navigated the increasingly complex waters of music-related gaming in an article over at Kill Screen.  Of course we’re not the only developer exploring this space and this is a nice survey of some of the latest developments.  It also brings some focus and clarity to how we talk about the aesthetics of “music games.”  Ultimately, that’s far more important than genre labels or snappy marketing taglines.

The article includes a couple of quotes from us on Thumper.

killscreen

Update! Dan has posted the full text of all the developer interviews on his blog.  It’s interesting to read how these inspiring and adventurous developers approach the same questions.  We’re psyched to be featured alongside them.

Starting a New Year by Looking Back

It’s 2014 and we’re hard at work on Thumper.  Recently, the game received coverage from IndieStatik, The Guardian, IndieGames.com, and PC Gamer — they’ve all mentioned Thumper as an indie game to look forward to in 2014.

We’ve been developing Thumper for a long time, and we’ve done a lot of experimentation along the way.  To start the new year, we’ll share some footage from our early prototypes.  Oftentimes game developers are shy about sharing this sort of thing for obvious marketing reasons.  Early prototypes usually look rough and you don’t ever want your game to look bad.   Your game will change a lot during development and showing early versions can create false impressions or expectations.  But prototypes can be a fun and interesting insight into the creative process, so we’re happy to share ours!

So, with the standard caveat that these videos do not represent the final gameplay, graphics, or audio of our game, here is footage from one of our first prototypes.  We developed a custom engine for Thumper and at this early stage, our renderer was limited to drawing solid lines.  That limitation led to a pretty cool vector art style.

Next, a slightly more developed prototype.  At this point, we were able to draw lines and solid color quads.  It was the pinnacle of my programmer artistry.  This prototype also shows our first experiments with path barriers and jumping.

Shorty after we made this build, I added support for drawing meshes, textures, and a basic lighting model to our engine.  So Brian (the Drool artist) was able to start developing our visual style and, sadly, my promising career as a programmer-artist ended.  We’ll share more of our prototypes and concept art in the coming months, but if you want a better idea of were Thumper is headed, check out our teaser trailer.

Fan Art

Time to sharpen your pencils, Droolers!  After the release of our debut trailer, we were pleased to find that our game has already inspired some top-notch fan art.  First, Jared from Ohio perfectly captured Thumper’s super-chill personality and understated coolness.

fan_art_thumper
 
And this anonymous artist nailed CrakHed’s raging party-animal spirit.
fan_art_crakhed

These are so good, we can’t wait for more.  To submit your fan art, scan and send to drool [at] drool.ws.  Include your name and location.  Remember, Thumper is chill, cool, and loves to take it easy, while CrakHed is always stoked, psyched, and ready to rage.

Soon, we’ll be starting a fan art gallery on this site.  So send us your chillest Thumper and most stoked CrakHed and we’ll post ‘em.

Recent Coverage of the Debut Trailer

crakhed

“…the demonic/cyberpunk rhythm/racing game of your dreams.” -Kill Screen

“…it already looks gorgeous.” – Wired

“The right reaction to watching [thumper] for the first time is to sit there, eyes wide and mouth open.” – Indiestatic.com

“It’s been too long since a good rhythm game stole my life.” – PCGamesN

“Thumper Has Crazy Beetles for Miles” – Giant Bomb

“Thumper’s fast pace and strange world caught our attention.” – Pocket Gamer

DROOL HANNO APPENA PUBBLICATO UN TEASER CHE VA BEN OLTRE LE NOSTRE PIU’ VERDEE ASPETTATIVE!!!  – Gioca Giuè’s Sunday Links

“Thumper features a sci-fi beetle on a fast track, dodging obstacles provided by the creepy-looking Crakhed.” -Polygon

PC Gamer… Destructoid… Kill Screen Cheat Sheet…

Thanks for helping spread the word everyone.  It’s been great to see all the reactions, comments and feedback.  Soon we’ll be sharing more about the tech, gameplay, graphics, and audio of Thumper.  What are you most interested in hearing about next?  Let us know in the comments.

Engine Tech: Easy Collaboration with Object URLs

I’m Marc, the programmer here at DROOL.  This is the first in a series of technical posts where I’ll go “under the hood” of our game engine and THUMPER.  These will be aimed at game programmers — non-technical DROOLers might need a quick toke before reading further.

We’ve recently started using URL “links” to our game objects that we can easily share via e-mail.  It’s simple and probably not a unique solution.  But it’s been useful and feels like a feature every engine should have.   It was quick and easy to implement, but the most time was spent working around some minor annoyances.

Our need for “Object URLs”

Brian (the DROOL artist) lives in Providence, Rhode Island, and I live in Seoul, South Korea.  Because of the distance (and 14 hour time difference), our opportunities for liquid communication via voice and video chat are limited and we often resort to e-mailing each other about development issues.

Like most game engines, our assets are divided into files, each containing multiple objects (meshes, animations, sounds, etc.).  For our first game, THUMPER, we’ve accumulated hundreds of files and thousands of objects.  During development, there could be an issue with any one of those files or objects.  For example, Brian might have a rendering problem and send me an e-mail like this:

SUP FLURY.  There is a rendering problem with bug_eyed.mesh in gameplay/resource/bugs.objlib.  I’m not sure if there’s a problem with material settings or a code issue?

Writing e-mails like this isn’t hard, but we send hundreds of e-mails to each other every month.  Even in this simple example, there’s opportunity for the sender to mistype file and object names.  And the receiver has to navigate to the appropriate file, search the file for the object, and open the object editor.  Wouldn’t it be nice to avoid this error-prone and manual labor?

Now we do, with “Object URLs” that directly reference our files/objects.  These links can be pasted into an e-mail by the sender and clicked by the receiver.  For example, Brian’s e-mail now looks like this:

SUP FLURY.  There is a rendering problem with http://drool.ws/gameplay/resource/bugs.objlib?obj=bug_eyed.mesh.  I’m not sure if there’s a problem with material settings or a code issue?

When I click that link, the DROOL editor is launched, the “bugs.objlib” file is opened, and the editor for the “bug_eyed.mesh” object is opened automatically.

Registering a Custom URL protocol

The first step is to register a custom URL protocol for your editor’s executable.  For example, the “http” protocol is associated with your web browser.  For the DROOL engine, we defined a custom “drl” protocol and associated it with our editor’s executable.  Full details for multiple operating systems are described here, but these quick steps assume you’re using Windows:

  1. Copy the text below into a text editor.
  2. Replace all the drl protocols with your own custom protocol.
  3. Replace C:\\path\\to\\your\\editor.exe with the path to your editor’s executable (don’t forget to double backslashes).
  4. Save the text as a file with the .reg extension (e.g. custom_url.reg).
  5. Run the file via the command line or by double-clicking in Explorer.  You’ll have to click through some security warnings.
REGEDIT4
[HKEY_CLASSES_ROOT\drl]
 @="URL:drl Protocol"
 "URL Protocol"=""
[HKEY_CLASSES_ROOT\drl\shell]
[HKEY_CLASSES_ROOT\drl\shell\open]
[HKEY_CLASSES_ROOT\drl\shell\open\command]
 @="\"C:\\path\\to\\your\\editor.exe\" \"%1\""

Since we’re a two person team, my “deployment” method was to simply check this .reg file into source control and tell Brian to run it on his machine.  Large teams probably already have a way to deploy scripts like this to every team member’s machine.  Large teams might also have multiple editors installed on each machine (for different source branches or projects).  For brevity, I’m skipping over these issues here.

URL Format and Parsing

Now that your URL protocol is registered, you can test it by typing a link starting with your protocol (e.g. drl://test) into your web browser.  You might have to click through another security warning, but your executable should be launched and the full URL will be passed to it as a command line argument.  Now your next coding task is to parse the URL into a file name and object, open that file, and open the object editor automatically.  I’ll leave the details to you, but if you already support double-clicking your editor files in Windows Explorer, then you’ve already done 90% of the work.

My URL parsing is bare bones to minimize overhead.  I just used standard C string functions to extract the file and object name and I don’t worry about properly encoding/decoding special characters (all our file and object names are alphanumeric without spaces anyway).  But I did follow the standard URL “query string” syntax so that if I do use a legitimate URL parsing library, it will be trivial to extract the values.  You can use any format, but I prefer something that is short while still being human-readable.  Our format is simple:

drl://relative_path_to_file?obj=object_name

Making URL Links “Paste-able”

At this point, I thought I was finished with URL protocols, but I discovered that unlike standard http:// protocol URLs, if you paste a URL with a custom protocol (like drl://) into most e-mail clients (like Gmail), they don’t automatically get turned into “click-able” links when you send the e-mail.  Of course, you can manually make links with custom URLs into click-able links by using your e-mail app’s GUI, but that is not quick and easy enough for me.  I just want to paste the link and send it!

After some research, I learned that custom protocol URLs aren’t automatically click-able in most e-mail clients for security reasons.  Unfortunately, the standard workaround is a bit gruesome.  The “done thing” is to use a standard http:// URL that sends the user to a webserver, then the webserver redirects to the desired custom protocol URL.  It’s sad to complicate our tool chain with a webserver like this, but that’s what we’ve done.

So now this URL: http://drool.ws/relative_path_to_file?obj=object_name

Gets redirected here by our webserver: drl://path_to_file?obj=object_name

Now everything works.  There are many ways to redirect a URL, but probably the easiest way is to use Apache’s mod_rewrite and a regular expression rule.  I recommend that you don’t do any actual parsing of your URL in your webserver, just do a simple find/replace and then redirect.  I prefer to keep all the parsing in my editor and involve the webserver as little as possible.

UPDATE: As commenter F Fulin suggested on #AltDevBlog, an alternative to redirecting is to use a standard, but out-dated, protocol (e.g. gopher://) instead of a custom one.  If your e-mail client automatically highlights links using a protocol (and you don’t need to use it for anything else) you can hijack it and avoid redirecting.

More Editor Integration

Once you have this much working, everything else is gravy.  Now that we can go from an Object URL to a specific file/object in our editor, the obvious next step is to make it easy to extract Object URLs from our editor.  When you right-click objects in our editor, the context menu has a “Copy URL” option.  Selecting that copies the Object URL for a particular object to the clipboard.

Copy URL
Object URL copying in action

Other Applications for Object URLs

The convenience of Object URLs has already paid off in other ways.  Like most game engines, when we detect an error, we print out an error message to standard output.  Including the relevant Object URL(s) in our errors makes them more useful and actionable.  Conveniently enough for me, in the Visual Studio Output window, http:// URLs are automatically click-able.

It’s typical for a large team to have a server that continuously grinds through game files and objects, potentially producing errors for each object based on certain validation criteria.  On a large project, you might have thousands of object-related errors at any one time.  From my experience, it’s challenging to manage this process.  Artists and other content creators might not appreciate the importance of these errors and it’s hard to keep the overall error count under control.  If you include Object URLs with these errors, I suspect that everyone on the team will appreciate the added convenience and will be more likely to fix their errors promptly.

This was also posted to #AltDevBlog.

New Screenshots

It’s been a while DROOLers.  We’re hard at work on our first gameplay trailer and other exciting stuff we’ll announce soon.  In the meantime, here are a few screenshots from the upcoming THUMPER trailer to tide you over.

narrow_tunnel_scrape (2)

zillapede (2)

haunted_entrance (2)