Talk:DO/PRJ
From Dark Omen Wiki
Rob 2008
Some ideas on the *structure*, if not the data of the TERR block.
Header seems to be:
TERR {
char[4] ID = 'TERR'
long SIZE_OF_BUFFER_NEEDED_BY_GAME (1)
long WIDTH_IN_TILES - same as in ATTR block
long HEIGHT_IN_TILES - same as in ATTR block
long NUMBER_OF_'SUPER_BLOCKS' (2)
long NUMBER_OF_OFFSETS (3)
long TOTAL_SIZE_OF_OFFSET_DATA (4)
long UNKNOWN (5)
}
then come NUMBER_OF_OFFSETS * (16 byte structure)
and then NUMBER_OF_SUPER_BLOCKS * (8x8 = 64 byte tile)
(1) I can calculate this, but not sure if this interpretation is correct. In any case, this is NUMBER_OF_OFFSETS*8 + NUMBER_OF_SUPER_BLOCKS*64 + 24.
(2) This is the number of 8x8 'super blocks' in the file. These seem to be 8x8 tiles with elevation data (or similar)
(3) The number of offsets is the (width in tiles / 8 ) * (height in tiles / 8 ), with one catch, which is that the width and height should be rounded up to the next multiple of 8 (I assume to accommodate a whole 8x8 block).
(4) This is NUMBER_OF_OFFSETS*16. Not figured the offsets out properly yet, but the offsets seem to be 16 byte structures made up of 4 components (although I haven't actually seen one with a value larger than the maximum value of a signed short). There might be other data aside from offsets here also.
(5) This might be an offset to the first 'super block'
I'll carry on having a look at this and see if I can get further info out of the files. Also, this is only really speculative at this point.
EDIT: It's definitely elevation data. I've been able to extract height maps from the PRJ files, although looks like there is some additional data, which I'm not sure what it is yet.
Ok, I've found a few errors in the above, so version 2 follows:
TERR HEADER
{ char[4] ID = 'TERR'
long TOTAL_SIZE_OF_ELEVATION_MAP
long WIDTH
long HEIGHT long NUMBER_OF_SUPER_BLOCKS
long NUMBER_OF_OFFSET_HEIGHT_PAIRS
}
then there follows an offset/height pair block {
long SIZE_OF_OFFSET_HEIGHT_PAIR_BLOCK
[ elevation map, including destructible buildings ] NUMBER_OF_OFFSET_HEIGHT_PAIRS *
{ signed long MIN_HEIGHT_FOR_SUPERBLOCK long OFFSET_INTO_SUPERBLOCK_DATA
}
[ elevation map again, this time not including destructible buildings]
{ signed long MIN_HEIGHT_FOR_SUPERBLOCK long OFFSET_INTO_SUPERBLOCK_DATA
}
}
next follows the super block data
{
long SUPERBLOCK_DATA_SIZE NUMBER_OF_SUPER_BLOCKS * {
byte[64] ELEVATION_DIFFERENCE_8x8_BLOCK
}
}
to get the height for each individual tile, you have to add the minimum height for the superblock to the difference for the associated tile.
Mikademus 2008
Ok, so basically TERR divides the map into a grid and specifies the terrain elevation of each cell and the elevation of each cell including any structures?
I guess for the editor I could automate generating a TERR structure from the terrain meshes and the INST entities.
Rob 2008
Something like that yes. The map is divided into 8x8 cells.
Looks like they probably got the information via a pre-processing step by using the mesh.
I assume that when you destroy a building, the engine just adjusts the appropriate pointer to the cell.
Rob 2008
I've managed to get my dark omen to work, ground and all!
I did some tests last night, and the ATTR block doesn't seem to affect where you can move, which is a bit of a blow. All that I've found it to do so far is to affect the animation that plays when a cannon ball hits it; when I changed the water to rock, I get yellow sparks instead of water splashes.
So the movement information must be stored elsewhere, or calculated on the fly by the engine, possibly from the elevation data? Anyone know of any areas where you can't get to but there is no change in elevation?
Mikademus 2008
You think it could contain the difficulty of terrain rather than a boolen moveabilty toggle? In the Tabletop (4th ed) rules, which DO followes quite exhaustively, there are differnt types of terrain restricting you to 50% and 25% of your unit's original move rate (unless the unit has the "unaffected by difficult terrain" flag).
Rob 2008
I'd like to be able to put up some images of what I get out of the ATTR and TERR blocks, so people can see a little more clearly what I'm talking about.
Not my bandwidth though
EDIT:
http://gallery.filefront.com/Zeroed/
you can see the dumps for the first map TERR block. First one is the 'difference tiles' second one is the cell minimums, third is the two maps combined (in a paint program) = elevation map (with structures). Last one is the ATTR block.
Rob 2008
Got bits of another block figured out. Seems to be the camera sweep at the beginning of the match. It's certainly some kind of path.
TRAC {
long NUMBER_OF_PATHS = 2
NUMBER_OF_PATHS *
....[
....long NUMBER_OF_PATH_HEADER_STRUCTS = 6
....long NUMBER_OF_POINTS_IN_PATH
....NUMBER_OF_PATH_HEADER_STRUCTS * [ 16 bytes (unknown) ]
....]
SUM( NUMBER_OF_POINTS_IN_PATH ) *
....[
....signed int X
....signed int Y
....signed int Z
....]
}
I haven't been able to work out what the EXCL or EDIT blocks are.
Mikademus 2008
Rob, really, really good analyses you've done! I'm impressed!
Now, given the nature of the TERR and ATTR blocks, I'm thinking about how to include and implement them in an editor. The current editor I'm doing to iron out the basic structures is in text mode (Olly can tell you more of it), but I think the two blocks can only really be done gainfully in graphics. Perhaps I can calculate them automatically, and allow manual adjustments in the 3D editor.
Rob 2008
Hmm, I was trying to get straight how the originals must have been made. Which came first, the mesh or the heightmap?
I think that the mesh must have come first, since the heightmap seems to contain heights for the furniture. Maybe they made the mesh, placed the furniture and then cast rays downwards to get height data?
The paths are in a very simple format, but it looks like it's preprocessed to me. Having a path made up of hundreds of points just isn't very editor friendly. I wouldn't be surprised if the unknown bytes are control points for splines, although I have no knowledge at all in that area. EDIT: on second thoughts, this could be in the 'EDIT' block
I'm also not sure what the second path in the TRAC block is. It looks like the first one is the camera track as I mentioned and I assume that on multi-player maps, this is swept out in reverse for the opposing team. Even the single player maps have a second path though.
Also, we're running out of places for things to be. It looks like pathing may be in the M3X file, and maybe deployment zones?
Mikademus 2008
According to Smurf, deployment zones are in the BTB file. Based on how I would implement it given the data it could be that pathfinding is done through the height map using the difference between cells as the cost in an A-star algorithm.
Rob 2008
Regarding pathfinding, I assumed that there would be some explicit graph somewhere, since it presumably isn't possible to travel directly between all nodes.
If it is implemented as you suggest, then that would imply that which areas are accessible is controlled only by the heightmap (i.e. a big change would mean an area is inacessable).
We can test this further I suppose, when we have the ability to modify the heightmap directly.
Mikademus 2008
It can't be just as simple as I only the heightmap, since then units should be able to move over the pond. So probably the ATTR is accounted for in the pathfinding node map. However, looking at the gradients in the height map, which seems to be very accentuated along the edges, it is very likely that too great difference means that no connections are made between adjoining nodes. This will be interesting to test!
Another interesting thing to test is whether units are Y-positioned (elevated) based on the mesh or the heightmap.
Rob 2008
Already tested the last one. Modified the first mission by blanking all of the cell differences, so I was left with an approximate heightmap. Units sank into the ground in places, as one would expect.
So the TERR block is definitely what's used for colision with the terrain.
Mikademus 2008
k, that's good, because that means we can write off the BASE and WATR blocks (or rather, the terrain and water meshes) as significant beyond mere decoration: for the game mechanics, the relevant blocks are TERR and ATTR. EXCL might be relevant too, though that guess is based solely on the block name which suggests EXCLuded regions.
Rob 2008
I looked at the SHD file today, and its another map encoded in cells like the TERR block. Maybe its a lightmap? SHD does sound like "shader" or "shadows", though I doubt they had what we call shaders today back in '98.If "shadows" it could be a lightmap, or a darkmap.
Mikademus 2008
I think I have figured out a bit more about the final three blocks: EXCL, MUSC and TRAC. I've mentioned above that the block size item of every block is inconsistently implemented, which to me suggested that they were leftovers from an earlier design model. This seems to be further validated by the fact that these three last blocks lack the size item altogether, and thus likely added to the PRJ file as the need arose but after when the change to ignore the block size value was completed.
Thus, when Rob and were trying to understand the logic behind the EXCL block's size item we were chasing a red herring. Studying the eight first PRJ files, the only values occurring in that position are 4, 8, 12 and 16, which could suggest binary attributes (4 = 100, 8 = 1000, 12 = 1100 and 16 = 10000). Another observation is that no EXCL block ever seems smaller than 32 bytes.
The MUSC block lacks size item and is always 20 bytes large (24 bytes including "MUSC").
Rob 2008
Well, there doesn't seem to be that much overall structure to the PRJ file. We're probably best just treating it as several concatenated files. :/
The SHD file does look like it might be a lightmap (or rather, darkmap) by the way. Large values around trees and the like. SHD probably stands for shadowing.
Mikademus 2008
I have investigated the "header" sub-blocks of the TRAC block. First, they are not quaterions. But they do seem to be 3 x angles + 1 x unknown, where the angles, for some bizarre reason, are saved as degrees rather than radians, as everywhere else. (Possibly rotations in Euler angles?) This could indicate that the tracks are direct exports from somewhere else.
Mikademus & Rob 2010
Mikademus> Rob, if you can derive an algorithm to encode height maps compatible with the PRJ/TERR format, then there's no problem
<Rob> It doesn't have to be hugely efficient, given modern machines memory. So just one tile for each 8x8 area. take the minimum height and store that, then store 1-diff tile per chunk.
<Mikademus> Sure, for DO even the crudest brute force solution will be fast enough for us, still I don't really relish designing the algorithm <Rob> Well, assuming that we can get an ordinary heightfield
<Rob> 1) Split into 8x8 chunks
<Rob> 2) find the lowest point in the 8x8 chunk
<Rob> 3) calculate the difference between the height and the minimum for each point in the 8x8 chunk
<Rob> 4) use the results of (3) to make a new 'diff' and use (2) as the minimum value
<Rob> 5) in the file, the diff blocks will look like 0,1,2,3,4,5, ...
<Rob> so no re-use. but like I said, we probably don't care
<Rob> anyway, if you want, you could try merging diff blocks afterwards
<Mikademus> Well, then you have that different height maps share macro data
<Mikademus> Two maps detail-encoded against the same source
<Rob> sure, make 'em the same first and see what happens
<Rob> even then, I doubt that they *have* to share data. Just put more copys of the blocks in.
<Mikademus> Let's hope it is that easy :) That't be convenient
<Mikademus> If we can make height maps then we have all the components for making new maps from scratc
Olly 2010
I have now created new Flat Terrain BASE.M3D models for any future conversion to M3X and adjustments to TERR and ATTR block. By increasing the Map Size, Battle Edge and Sight Edge in the BTB editor - troops can now walk off map but camera is restricted to original M3X. So by replacing Tutorial Map Spare9's BASE.M3X with Black Pyramid BASE.M3X it allows both Troops and the camera to roam off map but seeing how M3X is taken from its original M3D file, it draws pyramids.
So is it possible to create a new M3X file, based on my new flat BASE.M3D that i have created? As this should temporarily get around not having to edit the TERR and ATTR blocks in Spare9.PRJ since it is all flat. The Original map dimesions BASE.M3D for Tutorial is 18x18 and Pyramids is maybe 24x24 so ideally in the future I would love to make a 50x50 BASE.M3D and BASE.M3X
