Category Archives: XML

Kinda like JSON, but not as good.

Preserving Newlines in XML Serialization

With .NET’s XML serialization, it kills newlines when you serialize XML to disk. More specifically, it converts a CR+LF into just an LF (\r\n becomes just \n).

This was causing annoyances with the spell editor, since you can edit source code for spells with it, but the code would appear all on one line after saving and reloading.

The fix was to declare XmlWriterSettings with a newline preference.

The code was this:

         public void Save()
         {
             XmlSerializer serializer = new XmlSerializer(GetType());
             Stream stream = new FileStream(FileLocation.SpellDirectory + FileName,
                 FileMode.Create, FileAccess.Write, FileShare.None);
             serializer.Serialize(stream, this);
             stream.Close();
         } 

And now it’s this:

        public void Save()
        {
            XmlWriterSettings ws = new XmlWriterSettings();
            ws.NewLineHandling = NewLineHandling.Entitize;
            XmlSerializer serializer = new XmlSerializer(GetType());
            Stream stream = new FileStream(FileLocation.SpellDirectory + FileName,
                FileMode.Create, FileAccess.Write, FileShare.None);
            XmlWriter writer = XmlWriter.Create(stream, ws);
            serializer.Serialize(writer, this);
            stream.Close();
        }

I can have my newlines and eat them too. Oh joy!

Zone Format XML Schema

I used the Microsoft XSD.exe tool to generate an XML schema file from the zone classes. That probably means nothing to most readers of this blog, but here’s the .XSD for the Basternae 3 zone format:

Basternae 3 Zone Format XSD

No reason really, just in case someone might be an XML tinkerer and wants to try running it through some of their tools, or if some masochist wants to browse it. I’m pretty sure there are some tools out there that will generate a data editor based on an XSD file format, but I’ve never had my hands on one. I have heard that Java and Eclipse have some pretty good XML tools.

Fun With XSL

One of the good things about having data stored in XML files is that you can use XSL stylesheets (templates) to format the data into HTML.

I created two basic templates, one for races and one for classes, and linked them to the data files used by the MUD. They don’t have any fancy formatting, but they do display the data in a Human-able format.

The result is that the class and race webpages link to the data files being used by the MUD. This means that you can view info about each race and class exactly as it’s being loaded. If we change something in the data file, such as adding a new spell or adjusting a race’s dexterity, the web page that is displayed is updated automagically.

This means that the pages for each race/class will essentially maintain themselves. Neat, eh?

Artificial Intelligence Markup Language (AIML)

One of the things I plan to have the MUD engine support is AIML. That’s an XML-based file format that defines conversational data for a “chatterbot”. I’m sure most everyone has heard of the Eliza, Julia, or A.L.I.C.E chat bots.

The idea is that you can create personalities that respond to certain phrases, much like quest messages do/did in previous versions of Basternae, but you can take it even further by having compound triggers, matches, variables, and responses to previous inputs.

There’s really no limit to how simple or complex you can make your ‘bot’, from responding to only a single phrase to an encylopedia of knowledge that might almost pass the Turing test.

Here’s a bit of info about the format:
Overview from Dr. Wallace, creator of AIML

And, of course, here’s the obligatory Wikipedia link:
AIML on Wikipedia

One tool I’ve found that I like so far is the GaitoBot AIML Editor, which you can get here:
GaitoBot AIML Editor Homepage

If you know any German, the bot on the Gaito site is kind of fun to banter with. I know just enough to make it think I’m an idiot. 😛

If you want to start building your own bot personalities — grumpy Trolls, arrogant Elves, Dwarves too busy working to deal with your nonsense, or schizophrenic tinker gnomes — then feel to grab the GaitoBot app and start building personalities. The intent is to be able to load AIML files directly into the MUD and associate them with one or more mobs.

All I’ve done so far is generate some C# classes from the AIML schema, but that went smoothly enough. I’m not sure yet whether I’m going to build my own bot engine or use an existing open source one.

Removing Hard-Coded Rooms, Objects, and Mobs

The old codebase had plenty of references to hard-coded room numbers. For instance, “Limbo” and the “default start room” for races without hometowns were set to room number 200.

Well, having those etched permanently into code is a bad idea, since it ties you inextricably to certain specific area files, and more importantly, it forces “zone gods” to dig into code if they want to change some things with how rooms are used. Bad idea.

The interim solution I’ve come up with isn’t great, but it does at least take the specific numbers out of code. I’ve created some XML files that load at boot time: StaticRooms.xml, StaticObjects.xml, and StaticMobs.xml. They contains a list of room, object, or mob names and their numbers, storing them in a Dictionary type. This makes it easy to refer to “ROOM_NUMBER_START” or “ROOM_NUMBER_LIMBO” in code and not worry too much about where it actually points. Of course, those names are still hard-coded, but at least they can be administered by non-programmers now.

I’d like to have them load dynamically as part of the zones, but there’s a bit of a problem there — some bits are just far too tightly integrated, especially things that revolve around locations, such as shifting to the astral or prime material plane. Those problems are better tackled in a future design change. This incremental improvement is good enough for now.

More Fixes To File Loading

A week ago I wrote about Mono not being happy with XmlElementAttribute.  I had cleaned up loading/saving for players and zone files, but skill and spell data needed some work.  I took care of that today, so all of the xml serialization is happy now.

I’ve hired the wife as a QA tester to do some preliminary “how broken is it” testing so I know what needs to be fixed before the real work (building) can begin.

Mono 2.01 Doesn’t Like XmlElementAttribute

I have zone files loading on Mono now.

I had a sneaking suspicion that it had something to do with the XmlElementAttribute that I used to keep the XML tags in the zone files compatible with the encapsulated, property-based classes.  I spent a few hours updating the zone converter application so I could remove the XmlElement attributes.  After loading the MUD on Mono, all zone files magically loaded.  “All” being the ~6 that I’m officially authorized to use so far.

Since I’m running Ubuntu 9.04, the native version of Mono is 2.01, a bit behind the current version of 2.4 (which will ship with Ubuntu 9.10).  Installing Mono 2.4 on Ubuntu 9.04 is not as easy as one would hope, and I can’t say for sure that it would take care of the problem.  Mono has made great advances lately, so I’d be surprised if the latest version had the same problem.

Continuing Spell Migration

When we last saw our hero, he was working on migrating spells from being hard-coded in 15 different places to being individual XML data files that are loaded at boot time.

While this makes it easy to modify, enable, disable, rename, or adjust spells without a recompile, the main reason I’m doing so is because there’s just so dang much clutter in the codebase with spell info strewn about everywhere.  It’s like a pack of wizards exploded in there, I swear.

This will also make it far easier to make minor adjustments for balance.  Instead of having to track down a developer to go through many slow tweak-recompile-reboot cycles just to adjust spell damage, the “balance engineer” will be able to work independently.  In the future it may be possible to perform online editing of spell attributes without a reload.  This is obviously not something that everyone should be allowed to do.

Part of the migration will be to move most of the actual function code into the spell files.  Most spells are pretty generic in nature — add an affect (flight, strength, poison) or affect hitpoints in some way (fireball, heal).  Even so, the same ~10 lines of code are duplicated for pretty much every spell with only the name changed.

Making the spells somewhat independent of the game engine also makes the engine less “tied” to Basternae.  At some point I’d like to create a space-based MUD using the same code, and the spells would obviously be dead weight.

A Whole New Spell And Skill Engine

Well, maybe not a *whole* engine quite yet, but certainly two thirds of one.

I’ve generated about 500 little XML files to hold all of the skill and spell data for dynamic runtime loading.  It’s all pretty neat — skills and spells can be tweaked by hand without having to compile any code (though the MUD requires a reboot) and they’re loaded at runtime, stored in a Dictionary type, and accessed based on their names.

There isn’t a single hard-coded spell or skill value in the engine.  Hardcoded values were something that always bothered me.

Another thing I did in this process is embed “logical preference” data in the skill and spell files so that the AI engine can be improved while at the same time removing the need for thousands of lines of “spaghetti code” like in Basternae 2.  You see, each spell and skill check in B2 was hardcoded in a specific order and with a specific percentage chance.  Adding a skill or rearranging mob AI meant editing these in more than one place.

Instead (when the AI code is done), we’ll be able to set a few flags on each spell or skill and it will handle it automatically (unless a specific mob has a personality override file).

Oversimplified example:

Fireball.xml:  Type = offensive, Preference = 65, Likelihood = 40
LightningBolt.xml:  Type = offensive, Preference = 50, Likelihood = 50

This means, essentially, that a mob would have a 40% chance of casting fireball during combat, and if that didn’t go off, it would have a 50% chance of casting lightning bolt.  Changing the way the mob performs in combat is just a simple number tweak.

I have some pretty huge plans for mob AI, but this new spell/skill system goes a long way toward making those plans easy to implement.

I also have yet to embed all necessary custom code in the spell/skill files, but one thing that’ll be done is that most information needed to trigger a spell will be embedded in the file.  Instead of manually writing a dozen lines of the same code for each spell to validate the target, check saving throws, set damage type and amount, send messages, and deliver the affect(s), most spells will use one general-purpose function that checks the spell types and flags and executes the spell’s action.

This means that most standard spells that are just damage or single-affect, like “soulshield” or “fireball” won’t have any embedded code at all.  Instead, only super-involved custom spells like the enchantment spell “earthen smith” will have their own embedded instructions.

Well, that was long-winded.  It had to be — I’ve done a lot and I’m pretty proud of what I’ve accomplished with this.

Fraglist, Crime, and Corpse Files Converted

The fraglist, crime, and corpse data files have been converted.  The crime file conversion required no effort whatsoever since justice doesn’t do anything with it.

Right now I’m working on a scheme to replace the old typo, idea, and bug file concepts, which were pretty useless.  The idea was that anytime a player typed “bug <whatever>”, it would go into a file on disk that could only be accessed from a shell account.  In most cases, the bug, idea, and typo files were completely ignored.  With this new system, immortals should be able to read, fix, and modify any of these issues online.  I’m still working out the details, but it’s looking pretty good so far.

Help File Converted

The help entries have been converted to XML.  Instead of being embedded in the area files and potentially spread across any number of area files, they are all in a file that contains only help entries.  In practice, Basternae 2 only had two area files that contained help entries, but it’s pretty silly not to have them all in one place.  The idea was, I suppose, to be able to have area-specific help sections.  Nobody used it.

I’d like to work out a nice way to be able to switch intro screens.  For instance, if it’s October maybe we’ll want to show a Halloween-y screen and show the regular screen during the rest of the year.  As it is now, the screen file has to be hand-edited and the game restarted in order to change anything.  It would be nice to have a handful of screens that load automatically at boot time and be able to switch between them without restarting the game.  In fact, part of the goal with this incarnation of Basternae is to never actually need to reboot for any reason (providing the game is stable enough to never crash, and that’s one goal I’m aiming for).

Socials Converted

The socials file, a text file containing 127 KB of text for the social actions in the MUD, has been converted to the new XML format. The help entry file is next.

Converting Races by Hand

I spent quite a lot of time manually converting the race files to XML.  The file format didn’t lend itself well to automatic conversion, nor was much of it an easy search-and-replace setup.  Some of the values were strings of flags, some integers, some full sentences, without much consistency.

It wasn’t fast and it wasn’t easy, but the race files are now loading in the new Basternae.

Converting area files should be a lot easier since they have a more consistent format, but we have a lot to convert before we get to that point — class files, for instance, which will be very similar to race files.

XML Objects!

It’s done – objects save and load as XML data rather than some ad-hoc text format.  I’m sure there will be a few extra details to work out, but the saving and loading of basic objects works now.  The change cleared up a bug or two that would come up once in a while due to formatting inconsistencies.

I still have to fix the violence update problem that I mentioned on the 1st.  I’ve tried a few minor changes in the hope that the problem would be resolved, but it really does look like a full combat-process rewrite is in order.  The original method is fundamentally flawed, so we need to work out a better way.

More XML Conversion

I’ve started tackling the conversion of all object saving to XML. Player saving was easy, since players tend to be pretty much the same and have all the same data fields. However, with all the different types of objects, nesting, affects, extra descriptions, etc. objects are a bit more of a project to convert. Objects are also saved in more than one type of file — corpse data file, player files, storage chests, etc.

So far I have them saving to XML the way I want them to. The next step is loading, which will take a good solid afternoon or evening of codework.

I’ve been mulling over the idea of moving data files over to an SQL database rather than XML, since XML is DREADFULLY SLOW. It’s not dreadfully slow on this fast machine, nor will it be on any server that I use, but it could become an issue at some point.

SQL would be a good thing and a bad thing. I’d get the automatic field matching and data integrity at the expense of greater complexity. Although there are tools to edit SQL databases directly, it’s not as easy as editing a text file if I needed to change a value in a file. Backups would also be a bit more complicated.

One thing at a time — I need to finish this first.

XML Files

Traditionally MUDs I’ve seen have used raw text files for player files. Some might use binary, and some might compress them into a gzipped format.

There’s a problem with using this sort of file: in general, they are written or read a single line at at time. In Basternae 2 we had a significant problem with player file corruption. If the save or load engine had a problem with one of the values found in the file, you were pretty much screwed. If this happened during a file save, you pretty much had to restore the player file from a backup and hope that the backup was recent.

In order to banish this problem to the realms of oblivion, I’m incorporating the XML-based file save/load code that I used on my AlgoRhythmia 3 application. The difference with this code is that instead of opening the file and writing it line by line, dealing with values as they come, instead it creates an XML data store in memory and populates that with the values, and then serializes it to disk after all the values are checked.

So, intstead of having a half-written player file if the values are bad, it’ll be an all-or-nothing. If the data store fails to build, the file won’t be written, so instead of losing up to a week worth of play data, at most about 15 minutes would be lost, and that assumes the player hadn’t picked up or dropped any items in that time.

Once I have this code done I may give serious thought to setting up all of the game files to use XML. In addition to the build-then-save, XML also some built in checks and guards, so a typo in a hand-edited file would be much less likely to destroy the universe.

The downside: XML files are larger and slower.

The other upside: Data files written in XML can be easily processed by other applications, such as a web script that posts dynamic game stats on a page.