Talk:DO/The .PRJ Battle Project File Layout
From Dark Omen Wiki
Mikademus Replies:-
"Yes, I have used that source code.
However, a Dark Omen battle is defined in the .PRJ files.
They consist of 11 sections, of which his source only covers four
BASE mesh (the surface)
WATR mesh (water)
FURN files (uxtra 3D objects placed on the map) and
INST (the instances of the objects)).
That is, they can load and draw the map, however, the two largest sections of the project file are
TERR (presumable "terrain") and the
ATTR("attributes"?) blocks (and four others), and we have no info about them.
My assumption atm is that the
TERR block contains surface abstractions for pathfinding, elevation, traversability and difficulty, among other things. Also, the
ATTR block seems critical, but that is even more difficult to map".
Rob Replies:-
"I figured out a bit of the ATTR block:
ATTR
{
char[4] 'ATTR'
long BLOCK_SIZE
long WIDTH_OF_MAP
long HEIGHT_OF_MAP
}
Then follows (width*height/2) bytes. Each tile has 4 bits of attributes and these are packed 2 to a byte.
Then there are 64 bytes, of which I can't make out much.
It looks like this is line-of-sight / accessibility information. attributes are 0 for 'ordinary' tiles, 2 for blocked ( i.e. unit can't move there ), 8 for passable water (such as fords), 10 (8|2) for impassible water.
There are probably more attribute values, its just a matter of looking at all the maps. I made a program which spits out the data as a bitmap, so I can just compare to the minimaps".
Mikademus:-
"Excellent work. And good, solidly testable hypotheses. Have you already tried changing these values to see what happens in the game?".
Rob:-
"Unfortunately not yet, since I can't seem to get Dark Omen working on my XP machine; I get the dreaded black ground problem. I'll be able to test it better when I can set up a win98 virtual machine.
Having looked at the data again, I think that it's probably actually only accessibility info and not line-of-sight. Otherwise if you put the cannon at the top of cliff, then it would have no line of sight underneath the cliff.
I'll let you know more when I've got DO working on my machine and I can test better".
Mikademus:-
"Thanks! Looking forward to your reports!
As for the discussion, though I do not know this, I doubt that there LOS info saved. That is usually only done in BSP or portal-based environments, where it is quite advanced and involved (bordering on ray-tracing complexity). I doubt the LOS calculations in DO would be very demanding, but then again, the game was made in 1997-8 so they might have saved CPU juice where ever they could.
Something funny to be tested would be what different particle effects exists. If you look at the records in the INST block you'll find that there is one that doesn't use any model but has the particle effect flag set. It is the churning water at the base of the waterfall. If we change that we should get all the other particle effects in the game: smoke, fire, explosions perhaps. It would be really good to know what value that indicates a particular effect".
Rob:-
"Some ideas on the *structure*, if not the data of the TERR block.
Seems to be:
TERR HEADER
{
char[4] ID = 'TERR'
long TOTAL_SIZE_OF_ELEVATION_MAP
long WIDTH
long HEIGHT
long NUMBER_OF_SUPER_BLOCKS
long NUMBER_OF_OFFSET_HEIGHT_PAIRS
}
then there follows an offset/height pair block
{
long SIZE_OF_OFFSET_HEIGHT_PAIR_BLOCK
[ elevation map, including destructible buildings ]
NUMBER_OF_OFFSET_HEIGHT_PAIRS *
{ signed long MIN_HEIGHT_FOR_SUPERBLOCK
long OFFSET_INTO_SUPERBLOCK_DATA
}
[ elevation map again, this time not including destructible buildings]
{ signed long MIN_HEIGHT_FOR_SUPERBLOCK
long OFFSET_INTO_SUPERBLOCK_DATA
}
}
next follows the super block data
{
long SUPERBLOCK_DATA_SIZE
NUMBER_OF_SUPER_BLOCKS *
{
byte[64] ELEVATION_DIFFERENCE_8x8_BLOCK
}
}
to get the height for each individual tile, you have to add the minimum height for the superblock to the difference for the associated tile".
Mikademus:-
"Ok, so basically TERR divides the map into a grid and specifies the terrain elevation of each cell and the elevation of each cell including any structures?
I guess for the editor I could automate generating a TERR structure from the terrain meshes and the INST entities".
Rob:-
"Something like that yes. The map is divided into 8x8 cells.
Looks like they probably got the information via a pre-processing step by using the mesh.
I assume that when you destroy a building, the engine just adjusts the appropriate pointer to the cell.
I did some tests last night, and the ATTR
block doesn't seem to affect where you can move, which is a bit of a blow. All that I've found it to do so far is to affect the animation that plays when a cannon ball hits it; when I changed the water to rock, I get yellow sparks instead of water splashes.
So the movement information must be stored elsewhere, or calculated on the fly by the engine, possibly from the elevation data? Anyone know of any areas where you can't get to but there is no change in elevation?".
Mikademus:-
"You think it could contain the difficulty of terrain rather than a boolen moveabilty toggle? In the Tabletop (4th ed) rules, which DO followes quite exhaustively, there are differnt types of terrain restricting you to 50% and 25% of your unit's original move rate (unless the unit has the "unaffected by difficult terrain" flag)".
Rob:- "Yeah, I'd like to be able to put up some images of what I get out of the ATTR and TERR blocks, so people can see a little more clearly what I'm talking about".
You can see the dumps for the first map TERR block. First one is the 'difference tiles' second one is the cell minimums, third is the two maps combined (in a paint program) = elevation map (with structures). Last one is the ATTR block".
http://darkomenworld.freeforums.org/looking-at-the-data-files-t33-45.html
Got bits of another block figured out. Seems to be the camera sweep at the beginning of the match. It's certainly some kind of path.
TRAC {
long NUMBER_OF_PATHS = 2
NUMBER_OF_PATHS *
....[
....long NUMBER_OF_PATH_HEADER_STRUCTS = 6
....long NUMBER_OF_POINTS_IN_PATH
....NUMBER_OF_PATH_HEADER_STRUCTS * [ 16 bytes (unknown) ]
....]
SUM( NUMBER_OF_POINTS_IN_PATH ) *
....[
....signed int X
....signed int Y
....signed int Z
....]
}
I haven't been able to work out what the EXCL or EDIT blocks are.
Mikademus:-
"Rob, really, really good analyses you've done! I'm impressed!
Now, given the nature of the TERR and ATTR blocks, I'm thinking about how to include and implement them in an editor. The current editor I'm doing to iron out the basic structures is in text mode (Olly can tell you more of it), but I think the two blocks can only really be done gainfully in graphics. Perhaps I can calculate them automatically, and allow manual adjustments in the 3D editor".
Rob:-
"Hmm, I was trying to get straight how the originals must have been made. Which came first, the mesh or the heightmap?
I think that the mesh must have come first, since the heightmap seems to contain heights for the furniture. Maybe they made the mesh, placed the furniture and then cast rays downwards to get height data?
The paths are in a very simple format, but it looks like it's preprocessed to me. Having a path made up of hundreds of points just isn't very editor friendly. I wouldn't be surprised if the unknown bytes are control points for splines, although I have no knowledge at all in that area. EDIT: on second thoughts, this could be in the 'EDIT' block
I'm also not sure what the second path in the TRAC block is. It looks like the first one is the camera track as I mentioned and I assume that on multi-player maps, this is swept out in reverse for the opposing team. Even the single player maps have a second path though.
Also, we're running out of places for things to be. It looks like pathing may be in the M3X file, and maybe deployment zones"?
Mikademus:-
"According to Smurf, deployment zones are in the BTB file. Based on how I would implement it given the data it could be that pathfinding is done through the height map using the difference between cells as the cost in an A-star algorithm".
Rob:-
"Regarding pathfinding, I assumed that there would be some explicit graph somewhere, since it presumably isn't possible to travel directly between all nodes.
If it is implemented as you suggest, then that would imply that which areas are accessible is controlled only by the heightmap (i.e. a big change would mean an area is inacessable).
We can test this further I suppose, when we have the ability to modify the heightmap directly".
Mikademus:-
"It can't be just as simple as I only the heightmap, since then units should be able to move over the pond. So probably the ATTR is accounted for in the pathfinding node map. However, looking at the gradients in the height map, which seems to be very accentuated along the edges, it is very likely that too great difference means that no connections are made between adjoining nodes. This will be interesting to test!
Another interesting thing to test is whether units are Y-positioned (elevated) based on the mesh or the heightmap".
Rob:-
"Already tested the last one. Modified the first mission by blanking all of the cell differences, so I was left with an approximate heightmap. Units sank into the ground in places, as one would expect.
So the TERR block is definitely what's used for colision with the terrain".
Mikademus:-
"Ok, that's good, because that means we can write off the BASE and WATR blocks (or rather, the terrain and water meshes) as significant beyond mere decoration: for the game mechanics, the relevant blocks are TERR and ATTR. #ID:_.22EXCL.22_data_block_layout might be relevant too, though that guess is based solely on the block name which suggests EXCLuded regions".
Rob:-
"I looked at the SHD file today, and its another map encoded in cells like the TERR block. Maybe its a lightmap"?
Mikademus:-
"SHD does sound like "shader" or "shadows", though I doubt they had what we call shaders today back in '98.If "shadows" it could be a lightmap, or a darkmap".
Mikademus:-
"I think I just figured out what the EXCL block does. When it is zeroed out no furniture block LoS for missiles or artillery anymore, and if I did not mistake myself, all enemy troops were always visible (if you were turned toward them).
Anyway, no missile troop could get the arrow-with-eye cursor.
But I have no idea how this is determined. In B1_01 EXCL is 68 bytes for non-header data, but that battle contains ten FURNiture and 37 INSTances".
Mikademus:-
"I think I have figured out a bit more about the final three blocks: EXCL, MUSC and TRAC. I've mentioned above that the block size item of every block is inconsistently implemented, which to me suggested that they were leftovers from an earlier design model. This seems to be further validated by the fact that these three last blocks lack the size item altogether, and thus likely added to the PRJ file as the need arose but after when the change to ignore the block size value was completed.
Thus, when Rob and were trying to understand the logic behind the EXCL block's size item we were chasing a red herring. Studying the eight first PRJ files, the only values occurring in that position are 4, 8, 12 and 16, which could suggest binary attributes (4 = 100, 8 = 1000, 12 = 1100 and 16 = 10000). Another observation is that no EXCL block ever seems smaller than 32 bytes.
The MUSC block lacks size item and is always 20 bytes large (24 bytes including "MUSC").
Rob:-
"Well, there doesn't seem to be that much overall structure to the PRJ file. We're probably best just treating it as several concatenated files. :/
The SHD file does look like it might be a lightmap (or rather, darkmap) by the way. Large values around trees and the like. SHD probably stands for shadowing".
Mikadmeus:-
"I have investigated the "header" sub-blocks of the TRAC block. First, they are not quaterions. But they do seem to be 3 x angles + 1 x unknown, where the angles, for some bizarre reason, are saved as degrees rather than radians, as everywhere else. (Possibly rotations in Euler angles?) This could indicate that the tracks are direct exports from somewhere else".
