Bringing Skill Trees Back from the Dead


(lowlines - PS4) #1

So over the last week or so (at the request of @MentalMars) I’ve been working on my first real Borderlands project since I started making stuff to put up on lowlidev.com.au.The task was to bring the official Skill Tree builders back from the dead! :fist::skull:

Important: Some of this might lean into the more technical side of things. However I have tried to explain everything as best I can. If it gets too complicated, just skip past things and look at the pictures :wink:

With the release of each Borderlands game, Gearbox/2K created a section on the game’s official site that let you spec your characters and share your builds with friends. Unfortunately, over time if not maintained, stuff on the web eventually stops working, or worse, completely disappears when the next new thing comes along. Sometimes it’s just a matter of there no longer being someone around with the expertise to fix the issues that have sprung up over time.

Before I talk about what I’m actually doing, lets have a look at the current state of the official skill tree builders and why MentalMars and myself decided something should be done about it.


[ Brick’s skill tree from Borderlands 1 ]

Borderlands 1
The website got repurposed as a global hub for all things Borderlands. However with a quick google search you can still find the skill tree pages that are now stranded on some 2K data servers.

But hang on, they don’t load? That’s because in the last couple of years, Flash was abandoned as a valid medium for serving content on the web. Not only did it lack accessibility features, but it was being constantly hit with security exploits eventually led to many browsers choosing to completely block Flash content by default. It was also never supported by iOS, which was kind of the first big nail in the coffin back when iPhones came out. If you explicitly allow those pages to render Flash content, you can still use those skill tree builders, but that’s about it.


[ Salvador’s skill tree from Borderlands 2 ]

Borderlands 2
The official game site for Borderlands 2 is just gone. Fortunately there is this wonderful tool called the Wayback Machine, which allowed me to go back in time and rebuild those lost pages.

“Hang on,” you say, "those one don’t load!"
So the problem is here is that the Wayback Machine can only properly cache websites that have no ajax calls. When the skill tree builder is loaded, it pulls down a manifest file with links to all the assets it needs to render the page. These links are all hard coded. Another thing that happened with the internet in recent years was that not running everything via https (the little green padlock in the url) meant your content was insecure. That also goes double for loading files hosted on another domain. Modern browsers will just flat out block this kind of thing.


[ Athena’s skill tree from Borderlands: The Pre-Sequel ]

Borderlands: The Pre-Sequel
The Pre-Sequel was the most recent main Borderlands game to come out (2014), however the skill tree builder is actually a clone of the BL2 one with the characters swapped out. I don’t think you can navigate to these pages via the official site, but again with a bit of google searching, you can find them…

Hey! They load! But wait! If you try to choose your skills, it stops working…What’s going on here is another result of the Internet as a whole putting its foot down on the scourge which was Flash. Prior to recent advancements in HTML5, the only way you could play audio on a website was via a hidden Flash player that you could make calls to via JavaScript. That stuff gets blocked outright these days because there are a lot of other things a hidden Flash player could do and you wouldn’t even know about it! Unfortunately, whenever you click on a skill, it tries to play the click sound, and since that code is now blocked, the rest of the logic never runs breaking the whole skill tree builder! :broken_heart:


So now that we know what state the official skill tree builders are in, I am going to go through the process I did to bring them back.

First off, I wanted to get these up and running again. This required a fair amount of what is essentially called website scraping where I’d systematically iterate through everything and download a 1 to 1 local copy of the website files that I could then boot up on my local machine. You also have to fix any hard coded file references and of course disable stuff like that “playSound” hidden Flash player I mentioned for BL2/TPS. The Wayback Machine assisted quite heavily in recovering the majority of the Borderlands 2 assets, though thankfully most of them were actually still up on the 2K data servers so once I knew where to look, I was able to recover everything…Well ok, not EVERYTHING. Axton was the only character across all 3 games who was missing his manifest file (a JSON file with all of his skill tree data and links to assets that need to be loaded). So I had to rebuild that from scratch!

[ The landing page for the Borderlands: The Pre-Sequel skill tree app ]

Truth be told, I only ended up using a couple of things from these websites, being the backgrounds and the character name badges, which were both things I knew I would not be able to get once I started datamining the games themselves. However having working versions of all the official ones is handy because it allows you to see how someone else did it as that might impact how you’ll approach it in your own implementation. Oh and for Borderlands 1, the game is so old, that it was considerably more work to extract image assets from the game files, so I instead extracted the vector assets from the Flash skill tree builders to construct my own sprite sheets!


Despite Flash’s troubled history on the web, it’s actually used quite heavily in games to render user interfaces! It’s called Scaleform GFx and the long story short it is middleware software that enables someone with experience in programs such as Flash to build interfaces that can be rendered within a game engine. What this also means is that a lot of the assets are actually stored in vector format, meaning I would be able to extract stuff at a much higher resolution than I would if they were baked raster images. Unfortunately, the way it actually works is bit more fiddly. Some stuff is stored within the scaleform files as vector assets and other things are baked into raster textures that get loaded in at runtime. So some can be extracted out at a higher resolution and other stuff cannot. This is also the case with a lot of stuff in my Battleborn projects and i imagine will be the case in future Gearbox titles unless they decide to use a different piece of middleware to render their interfaces.

I’m still in the process of putting together all these assets, but here’s some screenshots of where I am at with it. The goal here is to build something that stays faithful to the original, but adheres to modern website standards (ie responsive, not reliant on third party plugins such as Flash to work) and has my own flavour sprinkled on top! Character renders courtesy of MentalMars!



I hope this wall of text was insightful look into something I am quite passionate about (and spent many very late nights putting it together). I hope to post the final app up on my website sometime in December, but I make no promises here! This stuff takes time, and often things come up that cause delays, but it’s feeling pretty good at the moment, which is why I wanted to share where I am at with it. :sunglasses:


Borderlands 2 VR, coming to PSVR December 14th!
(Is this thing on?) #2

Looks like an interesting project! http://bl2skills.com still exists, and has been the site of choice for posting BL2 builds for a long time. The official TPS skill trees still load for me on desktop, but man are they ever slow to start up.

The main deficiency of http://bl2skills.com is that you can’t see directly how different COMs affect the stats, which is one thing I did appreciate about the TPS one when it came out. Since that wasn’t even a feature in the skill tree tab in Borderlands 1, if that could be a long-term goal it would be awesome!


(lowlines - PS4) #3

Another long term goal I want is to be able to show the effects of character mods and “overclocking”. That’ll come after I put in the time and get a Borderlands version of the Battleborn Gear Search up and running. I actually haven’t got stats pulling through on the current build as I’ve been mainly focused on getting the core skill building functionality working. The official ones has vastly simplified the stat data and I probably wouldn’t be able to use it to do further calculations like the above mentioned without manually coding stuff.

I’ve seen there’s a few other community websites like bl2skills.com but a lot of the older ones are also starting to show their age or have been abandoned. Which makes me sad.


(Pandora's Industrial Engineer) #4

Even though I haven’t taken the time to try and learn this kind of stuff to gain a foundation, you still amaze me in how well versed and knowledgeable you are about data-mining and such. Not to mention that you seemingly do it out of and for passion alone.

An amazing job!


When you are done with the project, is there any way that I can have a copy of the TPS template to have for private use? Either as a webpage.html or whatever is the preferred method. As really all I need is the base template and the ability to change descriptions and add skills, I don’t even really need to be able to spend skill points or change the images, but it’d be nice.


Oh, and are you going to make the skill descriptions accurate, or are you just going to copy paste the in game stats? This is mostly or only a thing in BL1, where skills like Lilith’s ‘Girl Power’ says 2% shield at rank 1, but really it’s 1.5% I think.

But I know that could cause more confusion than help, so maybe not.


(Guajiro Pandoreño) #5

Nice work, thanks!


(Vault Hunter - Borderlands Addict) #6

Wow! :open_mouth: You are amazing. this is fricking awesome and furthers my point of how smart the forum members are in here. You all amaze me. This is awesome stuff right here. Kudos! :acmaffirmative:


(Pandora's Industrial Engineer) #7

I thought I’d give this thread a “poke” to see if there’s any update to your progress.


(lowlines - PS4) #8

There has been some, I’ve just had some things at work to deal with as of late that have limited how much time I can put into my personal projects. :slight_smile:

[edit]

#progress


(Pandora's Industrial Engineer) #9

Sorry to keep bugging this thread, but I really anticipate this project and its template. So I understand if you have other stuff you’re working on, but I going to at least make sure it’s in the back of your mind. :stuck_out_tongue:


(lowlines - PS4) #10

It’s actually going to be the first thing i work one after Battleborn Day is over. :wink:


(Pandora's Industrial Engineer) #11

My spidey sense is tingling, has it been 2 months? It’s kinda weird and a little freaky that I magically remember this thread again after 2 months without any reminders.


(lowlines - PS4) #12

Haha yes! Some things happened and I ended up focusing my efforts into some Destiny 2 projects. I will defintely get back to this project at some point! Maybe when @MentalMars starts chomping at the bit. :eyes:


(MentalMars) #13

Well @lowlines was one of the people who did a lot of work for Battleborn Day 4. We have worked together for a few weeks to organize a great event. We spend a lot of our free time into this project. After things wrapped up we decided to take some time off. Take things easy for a bit.

However we are still creators, so lowlines “casually” dived into his projects and I dived into mine. Just fun and passionate creation without deadlines.

The skill trees was also part of a Borderlands project i’m working on. The additional development time of “the highly anticipated game by 2K Games” (read Borderlands 3) also impacted the urge to have this project done asap.

Currently i’m exploring 2 projects of which the skill trees can be part of.
Project 1 - foundation ready, needs content.
Project 2 - is getting upgrades and in the “middle” of transitioning.

We both do these projects in our spare time, we both have full time jobs and a family that are also competing for our time :wink: sooo… work in progress


(Pandora's Industrial Engineer) #14

It’s fine with me. The 2 month thing is just when I usually remember this thread again for whatever reason. When it comes to free content, I never hold anything against an individual if they choose to take longer, or not do it at all. I just highly anticipate this project, so I sorta said I’d check in (every 2 to 3 months) for an update, and lowlines seemed to be fine with it.

If my earlier post came off as sarcasm, then I’m sorry, it wasn’t supposed to. It’s just me having fun with it.


(MentalMars) #15

No offence taken. I haven’t been very public with my projects. But it is great to see that you (and hopefully some other people) are looking forward to the things we create.


(Is this thing on?) #16

Well I for one certainly am! The stuff you guys put together for the last community Battleborn Day was really neat.

Now if only there was some way to persuade Gearbox to support you guys by increasing the Community Day Heads and Skins for some event or other…


(lowlines - PS4) #17

All good! Honestly, the more demand I get for something, the higher priority it will be for me to work on it. I like to share what I’m working on early, but that comes with the catch it might be a while until it is ready or I might even decide to abandon a project for various reasons. There is currently a lot of demand for Destiny 2 resources (and it’s also the game I am playing the most atm), so that makes it easier to work on stuff for it. I love Borderlands, but I haven’t any of the games in months and the last one came out a good while ago. But once Gearbox starts talking more about what comes next, you will likely see me shift my focus more to Borderlands etc. :slight_smile:


(lowlines - PS4) #18

I kept my word and have gotten back to working on the Skill Trees again. Apologizes to folks who have been waiting for updates on this, but this is the nature of development and it has been a very busy year for me!

I’ve been tweeting progress updates over the last week and I’ll try to remember to repost them here for non-Twitter folks!


Implementing History Web APIs


Adding Language Support



Making the Skill Trees Mobile Friendly



I hoping to have the Borderlands 2 / VR skill trees ready for the release of the PS4 VR port on December 14.


(What does it all mean!?) #19

Awesome Lowlines, if it’s anything like your Battleborn work, this will be super useful…

So I don’t quite understand the purpose of this project, are you re-doing BL2skills.com as an app or are you giving greater functionality to BL2skills.com? I’m pretty low tech so apologies if I’m just missing the obvious here…


(lowlines - PS4) #20

I’m building a tool that will eventually support all the borderlands games. There have been some quality community projects pop over the years, but when i started, most of them were dead or incomplete. :slight_smile: