Amazing Emulator Turns 2D NES Games Into Fully 3D Worlds

A strange new product appeared on Steam last week: a Nintendo Entertainment System emulator that translates the 2D pixel graphics of classic 8-bit games into 3D worlds built with voxels. It’s called 3dSen, it has standard and standard+VR versions, and it lets you experience a select slice of the NES library from unusual new perspectives.

3dSen didn’t just pop up out of nowhere. When creator Tran Vu Truc started the project over five years ago it was called 3DNes, and even the first demos turned heads. 3dSen is considerably more polished and usable now—so much so that Tran decided it was finally good enough to sell on Steam. (He also sells the VR version on Itch.io.)

3dSen is an NES emulator, so you need game ROMs to run on it. (Only the emulator is included in the Steam package.) It can run most NES games to a fair level of accuracy, but that’s hardly the main draw. 3dSen’s claim to fame is its ability to “3Dify” about 70 different games from the beloved console’s massive library.

That’s a relatively small number. The reason, as Tran told Kotaku via email, is because it’s actually a lot of work to convert all the graphics of even a simple NES game into voxels, polygons, and skyboxes.

“The amount of work really depends on the graphic complexity, gameplay duration… and if there is any 3D gameplay element in the game,” said Tran. “For example, simple games like Mario Bros., Dr. Mario, Donkey Kong, [and] Galaga only required several days of work, [while] games like Super Mario Bros. 3, The Legend of Zelda, [and] Excitebike (3D gameplay element) required weeks or even months to be created.”

A number of variables make the process difficult. For starters, adapting the 2D graphics of visually sparse 8-bit games to render in 3D is relatively uncharted territory. In many ways Tran has to make it up as he goes, on both technical and artistic levels. What’s more, it’s not always obvious how to “read” the chunky pixels of a low-res sprite.

“I often have difficulties to fully understand the pixelated graphics in NES games,” said Tran. “They have a high level of abstraction, so sometimes it takes a lot of time to fully understand what some graphic elements represent.”

In the project’s early days Tran experimented with automatic, algorithmic conversions. But while he considers that an interesting topic for theoretical future research, he quickly learned that only painstakingly handmade conversions could offer a pleasing level of sophistication and polish.

One reason is because 3dSen’s rendering engine offers an ever-increasing number of tricks for interpreting games’ flat bitmaps as 3D objects. For example, if you watch the emulator’s latest YouTube trailer, you’ll notice details like Super Mario Bros.’ title screen and suspension bridges morphing and bobbing smoothly, a Castlevania background skeleton swaying on its chains while casting a shadow, the Super Mario Bros. 3 royal palace’s floor extending “out” of the screen, and so on.

All of these touches exist solely because Tran took the time to carefully implement them on a per-game basis. You can see why a brute-force computer conversion wouldn’t prove compelling; translating NES graphics into 3dSen is very much an art. Another person could try and convert a given game and come up with an entirely different-looking result. (And in fact, they might have the chance to do that, since one of the project’s next major phases will be releasing the 3dSen Maker tool so that users can 3Dify games themselves.)

As many tricks as 3dSen has up its sleeve, some games present too many challenges to convert, at least right now. “Games with perspective/pseudo-3D graphics and/or 3D gameplay are very hard to be 3Dified with the current 3dSen engine,” said Tran. “3dSen will need a lot more work to be able to properly handle them.”

You heard it here first: The hardest thing to 3Dify is 3D itself.

Since it takes so much work to convert just one game, Tran has to pick and choose where to focus his efforts. Once he weeds out the technically difficult games, Tran uses several criteria to help decide which games to do next. These include how popular a game is on the internet, how often 3dSen fans request it, and how nostalgic Tran himself is for it. Tran plans to convert as many games as he can. Unsurprising, considering that the 3D visuals are 3dSen’s main attraction.

Another area that may get more attention in the future is alternate control methods. 3dSen’s VR version already supports standard VR controllers and offers what is by most accounts a very pleasant virtual lightgun experience. Potential exists for adding motion-based punching to a game like Punch-Out!!, for example, but that’s on the backburner for now.

3dSen is the rare emulator that focuses not on accuracy or compatibility, but a completely unique feature all its own. While translating 30-year-old NES graphics into 3D voxels probably won’t spark a larger trend, the best 3dSen conversions really are neat to look at and play with, and people seem to dig the idea of experiencing childhood favorites from a new perspective.

3dSen Maker’s release will add some exciting creative potential, too.

“Please keep in mind that we can have many profiles with different 3D interpretations for each game,” said Tran. “It will become interesting to see different 3D interpretations from different people.”