View source for Guide to Battlemap Modification*

{{RTW M2TW Modding Logo}} 
This article is based on [https://www.twcenter.net/forums/showthread.php?t=55785 Muizer's Guide to battlemap modification] the original article was written by [[Muizer]] and imported with his permission as part of the [[Scriptorium]] project.  The article may have been altered since import - please refer to the history page for details of changes, or reference the original document.

'''Notes:''' 
*'''Warning concerning hex editing'''
This tutorial is very old dating back to the days of using a hex editor to edit descr_geography. There may still be some relevant info in the pertinent sections of this tutorial, but for editing, you should use '''[https://www.twcenter.net/forums/downloads.php?do=file&id=2870 Squid's editor]'''.
*The article is based on the RTW file, the M2TW file is similar and can be regenerated from text file or edited with '''[[IWTE]]'''.
For related subjects see also:
*[[:Category:RTW Battle Map Modding]]
*[[:Category:M2TW Battle Map Modding]]

{{TOC right}}
==Introduction==
This guide concerns the structure and modification of the battlemap terrain as found in campaign games and custom battles.
[[Battlemap]] terrain is of course based on the input of the map tga's specifying heights, ground types, climates and features. This is the typical domain of mappers. However, these maps do not account for the levels of detail encountered on the battlemap. Not by a long stretch. A great amount of detail is added by the game in the translation from tga to the actual battlemap.

To illustrate this, I've selected a view from the [[RTR]] 7 map I've been working on. In figure 1 the view is shown with the original translation of vanilla. To illustrate how much of the terrain texturing can be attributed to ground type and climate, Figure 2 shows the same view with homogeneous ground type (sparse forest) and climate (test climate). This is as "dull" as the battlemap can get given the original height and features maps. As you can see there's plenty of "unexplained detail" left.


[[File:MBGFigure1.jpg|300px]]
Figure 1


[[File:MBGFigure2.jpg|300px]]
Figure 2

Next I stripped away as much of the relief detail " gained in translation" as possible. The result is shown in figure 3. To put the crown on my destructive work, I also removed the game's own contribution to texturing the battlemap and replaced it with "legend" colours. In figure 4, Blue is used to depict winter colours, while the summer colours are bright green.

[[File:MBGFigure3.jpg|300px]]
Figure 3

[[File:MBGFigure4.jpg|300px]]
Figure 4

Those in the know will instantly recognize the relevance of this image for Climate modification: The foreground (extending a little beyond the actual battlefield) has retained its textures. These textures are found in the summer and winter subfolders in the Data\Terrain\test climate folder. Without modifying the "translation" alternative or modified textures can be fed into the game. Problem is that the colouring in the distance remains unaltered. The result is typically embarrassing when you try to remove winter snow from a climate. That is, in fact, the original stimulus for this work: Vanilla offered insufficient variety in snow-free climates for a map that extends as far south as RTR7's.
Next up: How to modify the translation

===File listing and content===
To Modify the battlefield terrain I use the following files. Please note that it is important to preserve the originals of both as backups.

'''[[Descr_geography.db]]'''
The parameters governing campaignmap to battlemap translation are included in this file. Attempts to edit this file with a text editor will invariably fail, producing textureless battlemaps with see-through floors at best and a failure to load the game at worst. The file can, however, be modified using a hex editor. The method is described below

'''[[Descr_geography.txt]]'''
The content of this text file corresponds to that of the descr_geography.db. The game does not read the txt file though. Because the .db version only contains numerical values and path names and is devoid of explanatory text, the txt file is a useful reference. It should be used to keep track of changes you make.

The content of both files is shown in figures 5 through 7. When reading and editing the .db file take note that

* The text in grey is not present in the db file
* The text in red is in IEEE single (32 bit) format in thd db file
* The text in blue are characters in the db file
* In the .db file, each path name pointing to a texture is followed by a Blue-Green-Red colour code (BGR) in byte format
* The text in green is represented in the .db file by row ([0..5]) and column ([0..5]) numbers respectively that refer to the table in box 6. These numbers are in byte format.

I will discuss the meaning of each of these parameters in the sections dealing with ground type and relief modification.

[[File:MBGFigure5.jpg|300px]]
Figure 5

[[File:MBGFigure6.jpg|300px]]
Figure 6

[[File:MBGFigure7.jpg|300px]]
Figure 7

==Tools and their use==

Downloading and installing the Hex editor
As said before, the tool I used to edit [[descr_geography.db]] is a hex editor. There are plenty of them around on the web. The instructions on use in this guide are specifically intended for the XVI32 editor. Download and installation instructions can be found at

http://www.chmaas.handshake.de/delphi/freeware/xvi32/xvi32.htm

Before you open descr_geography.db, remember to make a backup. Open [[descr_geography.txt]] in a text editor (or import it in a spreadsheet). Make a backup of this file as well.

Setting up the Hex editor for use with the descr_geography.db file
In preparation to editing the file, I would recommend you change some of the display options.

* From the menu "tools" select "options":
* click the "Appearance" tab and make sure the boxes "Text Grid on the left" "show gridlines" and "Use Blank to Display Control Characters" are ticked.
* click the "Data Inspector" tab. Verify that the radio button is set to "little endian". In the area below it, uncheck all boxes except "byte (+)" and "IEEE single (32 bit).
* Go back to the "Tools" menu and verify that "Data Inspector" is ticked.

==Inspecting descr_geography.db==
You are now all set to inspect the geography.db file: open it from its current location. To find your way around the file, select the gridcell at the top left corner of the text window. Now use the cursor to move 4 cells to the right. In the data inspector window the IEEE single entry should be a value close to 0.01. This corresponds to the first value in the descr_geography.txt file (Figure 5, box 1). Note that entries in IEEE single format take up 4 gridcells: the next value (0.25) appears once you move the cursor another 4 gridcells to the right.

==Editing descr_geography.db==
When editing the file adopt the "golden rule": do not add or delete grid-cells and only replace gridcell content without altering its format. In practice, check that editing mode is set to "overwrite". Text can be typed in in the text window without further ado.
The procedure to alter the "machine coded" numbers is a bit more involved:

* when selecting a gridcell, make sure the value you want to change is displayed in the Data Inspector
* in the "tools" menu, select "encode number"
* check the radio button next to the required format
* enter the number in the text box
* press "OK"


==Recommended approach to battlemap modification==
There is a considerable overlap and interaction between the parameters set in descr_geography.db. Parameters specified in the first section (figure 5) apply to all combinations of climate and ground type. These global parameters act as constraints on the local parameters specified for climate/ground type combinations (figure 7).
In both global and local sections a distinction is made between parameters governing small scale relief (figure 5 box 1 and figure 7 box 8 ) and those governing ground type variability (figure 5 box 2 and figure 7 boxes 7 and 9). In all, a top to bottom, left to right approach isn't very useful, both for this guide and for modification itself. Instead we opted for a reconstruction of the "stripped down" landscape shown in figure 4. The modification of ground types are slightly simpler to explain and is treated before the modification of small scale relief.

==Modifying terrain texture distribution==

===Stripping it down===
To create the stripped down version of the battlemap ground types that serves as a starting point we altered the following parameters:

Global section (Figure 5, box 2):
*slope flat = 100
*slope factor flat = 0
*slope and curvature factors = 0 (not required)


As a result, the game will consider all terrain with gradients less than 100 (the entire map) as flat. By setting the slope factor to 0, there will be no slope effects in flat regions (againthe entire map).

Local section (Figure 7, box 9) :
* set the median value to 5 (never lower)
* set all other values to 0


In this example, I also stripped the small scale relief effects. See the paragraph on modifying small scale relief for instructions. In addition I assigned "legend colours" to the summer terrain by editing the BGR codes following their respective path codes. The legend is as follows:

* combined winter colours - blue
* rock and gravel - red
* sand - yellow
* Mud - pink
* grass of all types - light green
* shrub - cyan
* forest - dark green


The result is illustrated in ''Figure 4''. As you can see, the Sparse Forest summer textures indicate '''uniformal grass''' coverage. Odd as this may seem for a ground type "sparse forest" this is indeed the intended effect. In figure ''Figure 7'' box 7, we find that sparse forest has a hardness "shale" and quality "normal". When we consult the table shown in ''Figure 7'' box 6, we find that the ground type distribution is centered on a "grass_long" gridcell. Remember that in the .db file rows and columns are numbered from 0 to 5, so the coordinates of this gridcell are 4(row) by 3 (column). These are indeed the numbers (in byte format) found for Sparse Forest in the .db equivalent of Figure 7, box 7. Beware that different climates can have different "lookup tables"! The tables themselves, it seems, cannot be modified.

===Adding random noise===
To bring some variety to this monotonous landscape we must add some "noise" to the texture and colour distribution. What What!? Well think of it this way: adding noise means that not all tiles will be filled with "long grass". The more noise we add, the more tiles will have the textures corresponding to adjacent gridcells in ''Figure 7'' box 6. I suspect (but am not certain) the noise distribution is gaussian ("bell shaped).
The place to start when adding noise to texture distribution is the standard deviation entry (last column in ''Figure 7'', box 9). As long as this remains 0, you can do what you like but the terrain will remain homogeneous "long_grass". ''Figure 8'' shows the result when we set this value to 0.5. The effect of further increasing the standard deviation (to 1.5) is shown in Figure 9.

[[File:Figure8.jpg|300px]]
Figure 8

[[File:Figure9.jpg|300px]]
Figure 9

===Spatial component of noise===
At the moment, the terrain looks something most likely encountered in a bad LSD trip. All ground textures and colours can now be found on the battlemap, but they are a) completely unrelated to their "natural" position in the landscape and b) scattered all over the place like "static".
Addressing the latter issue first we now turn our attention to the "fade out" entry (''Figure 7'', box 9 third column). In ''figure 10'' I went back to a standard deviation of 0.5 but inserted a fade out of -0.1. Unlike the "fade in" entry, "fade out" has a profound impact on the distribution of textures. In short, the more negative the value is, the more "clumped" the seems to become. In the process, the range of textures also decreases. Setting the fade out to -0.2 just about brings us back to square 1 (''Figure 11'').
Besides increasing the standard deviation, another means of regulating the range of textures found is by increasing the "median" (''Figure 7'', box 9 second column). The result of setting the fade out to -0.1 while increasing the median from 5 to 10 is shown in Figure 12. This result is nearly (nut not exactly) the same as the one shown in ''Figure 8''.

[[File:Figure10.jpg|300px]]
Figure 10

[[File:Figure11.jpg|300px]]
Figure 11

[[File:Figure12.jpg|300px]]
Figure 12

===Slope effects===
Up to this point there is no marked link between textures and position in the landscape. This is not surprising, because I earlier instructed the game to consider terrain with gradients up to 10 (all the map) as "flat". So, we turn our attention to the Global section once more (Figure 5, box 2) and make the following changes that will make the game consider the entire map as "steep":

* slope flat = 0
* slope_factor_flat= 0 (not required)
* slope_factor_steep = 3
* curvature_factor = 0 (no change)


The result of loading the game with these settings is shown in figure 13. Steep areas now have rock textures with some mud interspersed. The footslopes and hills are covered mostly by grasses, though shrub and forest textures cause some mottling here. The gradient coincides with the sequence of texture types specified in the fifth row of the table in figure 7, box 6. Textures found in steep areas are found on the left, those found on flat slopes on the right.
It really is only now that the use of the "fade out" entry becomes apparent. Figures 8 and 12 may look alike, but they would not had the slope factor been applied to it. In my experience a large, standard deviation, coupled with a strong fade out results in a much more zonal distribution than a lower standard deviation without fade out would.
The same mechanism applies to 'Curvature" no doubt. I am not very impressed with this function's ability to mimick natural patterns of ground cover distribution.
A realistic change of ground cover with gradient is not linear. You may wish to reduce or remove the influence of slope effects. You can cap the influence of slopes in the global section (Figure 5, box 2) in both flat and steep areas. Figure 14 shows the result of

* slope_flat = 0.1
* slope_factor_flat = 1

[[File:Figure13.jpg|300px]]
Figure 13

[[File:Figure14.jpg|300px]]
Figure 14

===Modifying small scale relief===

Stripping it down
As with the ground cover, I started with a version of descr_geography.db stripped of any additions to the small scale relief. The three main sources of small scale relief are:

* a globally defined "erosion" pattern that divides slopes in up to three sections: hills and footslopes, talus cones and escarpments
* a locally defined additive component
* a mixed globally and local defined noise factor similar to the one employed to generate the ground cover variations

To remove or neutralize these I made the following changes
Global (''Figure 5'' box 1):

* smooth_ratio_min = 0
* smooth_ratio_max = 0
* slope_flat = 0
* slope_steep = 10
* slope_steepest = 10

Local (''Figure 7'' box 8)

* median = 5
* all other parameters = 0

For this section we decamp to a himalayan river valley. The result is shown in ''Figure 15''.
In the following paragraphs I'll be adding the three factors back in one by one to demonstrate how the terrain is constructed and how the parameters in descr_geography. txt affect the outcome.

===The erosion component===
The erosion component is called that (by me) because the effect is similar to the technique of "thermal erosion" often used in VR landscape generation. "Weathering" would be a more appropriate term, because the process mimicked is that of rocks breaking off steep slopes under the influence of frost action and chemical weathering and piling up at the bottom of escarpments forming so called "talus cones".
In RTW the erosion component is implemented in the global section (Figure 5 box 1). First, to allow any small scale relief at all, we must widen the interval between minimum and maximum noise to step size ratio. For instance:

* smooth_ratio_min = 0
* smooth_ratio_max = 1

A word of caution: this part of the guide is based on experiments I did. I think what follows is correct, but I offer no guarantees

Next we turn our attention to the areas of the battlemap unaffected by erosion, the flat regions. With the value set to 0, there aren't any yet. That leaves us with two parameters to deal with steeper slopes. These are split into two sections:[list][*] slope_steep defines the "nominal slope of steep regions": the less steep slopes found at the bottom of escarpments[*] slope_steepest defines the "maximum gradient of the terrain": the upper part of the slope (the escarpment).
Depending on the height difference that needs to be bridged, a varying proportions of the slope will have the nominal and maximum gradients. Only when the entire slope having the maximum gradient is not enough to bridge the "step size" dictated by the height map will slopes steeper than the maximum gradient be used. I set the steep slope parameters as follows

* slope_steep = 0.5
* slope_steepest = 100

The result is shown in figure 16. As expected there are no real footslopes because all terrain is either "steep" or "steepest". We can see that the valley side has a "stepped" appearance because slope_steep is at a low angle and slope_steepest is as close to vertical.

Footslopes are introduced onto the scene by raising the parameter slope_flat:

* slope_flat = 0.62

The value was chosen because it corresponds to an angle of 32 degrees: the stable angle of piled loose materials. The result is shown in Figure 17.

[[File:Figure15.jpg|300px]]
Figure 15

[[File:Figure16.jpg|300px]]
Figure 16

[[File:Figure17.jpg|300px]]
Figure 17

In a sense, fluvial erosion is also accounted for in the global parameters section. The values shown in ''Figure 5'' box 3 define river(bed) shapes. The first two are of direct interest to terrain modification. Both lower the river bed, the first multiplies its altitude by a fraction, the second adds an absolute height in meters. ''Figure 18'' shows what happens to our river valley when we set the fraction (valley factor) to 0.75. A larger height difference now needs to be bridged: the low angle footslopes have disappeared and the contribution of "steepest" slopes has increased at the expense of the nominal "steep" slope.

[[File:Figure18.jpg|300px]]
Figure 18

===The additive component===

This is where things get a bit bizarre. The additive component is set locally (figure 7 box 2 column 4 under the header "bias") for each climate/ground type combination. The effect of entering a "bias" is shown in figure 19.
Understanding the "pincushion effect" in flat areas requires a more profound knowledge of the program than is really necessary for modding purposes. In any case I do not claim to fully understand what is going on. I can hazard a guess though. If you're not interested in such speculation you can skip to the next section!
You will have noticed that the resolution of the battlemap is much higher than is accounted for by map_heights.tga. This coarse grid is subdivided into many smaller subpolygons. A procedure commonly used in virtual terrain modeling for generating the additional wireframe posts is midpoint displacement. An explanation of the process can be found here. In short the principle is that a new point C is created halfway between known points A and B. Its height is the interpolated value +- an amount that is a function of the height difference between A and B and a random factor. The sections AC and CB are likewise subdivided, and so on.
If this is indeed what is happening, then the holes in the pincushion terrain would mark the position of the "height posts" on map_heights.tga and the bias component (as presented in Figure 7 box 8) applies to the first subdivision. (I have long thought that a closer control of the process could be gained through editing [[descr_terrain_generation.txt]], but I am not so certain now. It could be one of the many remnants of aborted approaches scattered found in the files).
Not surprisingly, in Vanilla the bias is only used for groundtypes typically assigned to mountainous areas where both the height differences on the heights map and the resulting noise (see below) drown out the geometrical regularity of the effect. Nevertheless, this setting can be quite useful to spice up terrain characterized by features too small to appear on map_heights.tga.

[[File:Figure19.jpg|300px]]
Figure 19

The noise factor
Already the terrain looks fantastic!.........or rather it would to someone looking at computer game scenery for the first time since returning from a 15 year non stop spell observing Lemurs in the heartlands of Madagascar. It looks distinctly "geometrical" to me. I think we'd both miss the noise (albeit for different reasons).
The process of adding noise is pretty much identical to the one described for ground cover.

* The standard deviation governs the noise frequency distribution. The main difference is that the standard deviation is provided in meters. In figure 20 we can see the effects of increasing this value: the pincushion pattern is largely obliterated. Remember that we are only using one ground type (sparse forest). Using a variety of ground types will disrupted the pattern beyond detection.
* The median value governs the noise (or height?) amplitude.
* The fade out value governs the spatial distribution of noise. A negative value has the effect of smoothing the small scale relief, as is shown in Figure 21


The Global section (Figure 5, box 1) does impose constraints on the amount of noise added to the "stripped" relief. The interval [smooth_ratio_min, smooth_ratio_max] scales the noise function by step size:the steeper the slope, the larger the magnitude of the noise applied to it during subdivision.
The interval [step_min, step_max] constrains the combined effects of noise and "erosion". It is advisable to keep the upper setting below 2.5. A higher value could cause the elevation data present in map_heights.tga to be drowned out by the (then no longer) small scale relief generated by the game.

[[File:Figure20.jpg|300px]]
Figure 20

[[File:Figure21.jpg|300px]]
Figure 21

==Implications for mapping==
===Modifying climates===
Need I say more? The ability to remove snow ground cover in winter opens up the possibility of stretching the climate zones and extending maps further south without ending up with endless stretches of "meditarranean" climate.

===Coordinating height and ground types===
Even if one does not decide to modify [[descr_geography.db]], there are some important conclusions to be drawn about coordinating height and ground types. Most important of all it confirms what experienced mappers knew already: that slope is the major factor governing the appearance of ground types on the battlemap and should be used to coordinate ground types and height.
The ground type names "high mountains" and "low mountains" are especially misleading in this respect, suggesting a link with altitude instead. More appropriate would have been "mountain ridges" and "mountain slopes". Figure 22 demonstrates the need to coordinate ground type and slope: the small scale relief settings look quite natural on the footslopes, but for the cliff face the amount of noise is insufficient to disguise the geometrical pattern the terrain polygons. Ideally, the cliff face should be associated with a ground type characterized by a wider noise frequency distribution (standard deviation) and amplitude (median).

[[File:Figure22.jpg|300px]]
Figure 22

A modest attempt to modify the test climate ground cover and small scale relief is shown in Figure 23. I have striven, above all, to achieve a more coherent ground cover-relief relationship and I dare say I have made some improvement when the result is compared with the equivalent battlemap generated with the vanilla [[descr_geography.db]] shown in Figure 2

[[File:Figure23.jpg|300px]]
Figure 23

==Explanatory Model==
It's been a while since I posted this tutorial. Apparently modifying descr_geography.db is still a bit daunting. In fact I can't claim to fully understand how the landscape is generated, but I can present a model Ifind useful to anticipate and explain what modifying descr_geograpy does. 

[[File:Descr geography2b.gif]]


A single point on the map yields a climate-ground type combination. There are quite a few of them, but here I've followed one particular combo. Let's call it "low fertility ground type of the infertile grassland climate".

Each such a combination is centered on a battlemap ground type. In this case it's "shrubland". This is what you can expect to find predominantly on level ground. A noise factor (standard deviation) means also the groundtypes on either side can occur (short grass and forest). On slopes, things are different. The steeper the slope, the farther the whole distribution gets shifted towards the left.
That's how variety of battlemap groundtypes is roughly generated.

Descr_geography also changes and interpolates between the height map elevation. The map provides an very coarse grid of elevations connected by linear slopes. To this, descr_geography adds an offset, which actually raises some points, not the landscape as a whole, hence the curvature. It also applies a fractal function of the slope, causing the surface to be "rippled" in the direction of the height map slope. Finally random noise is added. The resultant (I believe) is the basis for the slope factor that affects the distribution of battlemap groundtypes.


Again, I'm not saying this is exactly how things work. For one thing, there's not just a slope factor to determine the impact of the terrain releif on battlemap groundtypes, but also a curvature factor, which operates along similar principles. And then there's the "fade in" and "fade out" that govern whether ground types occur clumped or scattered. Nevertheless, I think it's a useful model if you want to predict the effects of your changes. I hope this helps illustrate how things hang together and above all, why you should co-ordinate your height (or more specifically slope) and groundtype maps for the best results. 


{{permitted|link location=[https://www.twcenter.net/forums/showthread.php?t=55785 TWC Tutorial]|Name=[[Muizer]]}}

[[Category:RTW Battle Map Modding]]
[[Category:M2TW Battle Map Modding]]
[[Category:RTW Modding Tutorials]]
[[Category:M2TW Modding Tutorials]]