ANNOUNCEMENT: new tool for map making for Exult Studio

[artaxerxes]

To all people wanting to make their own maps for Exult Studio:

As you already know, there exists a tool to transform a 192x192x8 indexed image (BMP,PNG,GIF) into a u7map. This tool has been renamed mockup (used to be called mock_up) and it can be accessed at the following address: http://si-french.sf.net/mockup.tar.gz

One complaint I heard was that once you created the map, there was still a lot of work to do on it (Ram Dragon will know what I am talking about). For instance, to make a forest, you would have to either place each pixel manually in the image, or retouch the generated map so that the forest does not look like a big block of identical chunks. Another example would be to make the water and the land join each other smoothly (water chunk intermediate chunk land chunk) instead of a clear cut (water chunk land chunk).

Today, there is now a tool to do that automatically. This tool, baptised "smooth", takes as input an image file, generate as output another image file, in which changes were made according to a configuration file you created beforehand. All left to do is to feed this newly created image into mockup to get a proper u7map.

Smooth uses a plugin technology, so that everyone is invited to write plugins to create certain effects. For instance, up to now, only the randomize plugin as been created. It is loaded only if it is called for in the config file, just like all the other plugins would.

An example for the input image and the config file are available in the source code. To give you an idea of what the config file looks like, here is an extract:
-------------------------------EXTRACT------------------------
# comment
; another comment
# plugin names are in bracket and load libsmooth_randomize.so in this case
[randomize]
# this will replace every occurance of 323232 by one of 010101, 731632 or ae3b1f chosen randomly.
323232 010101 731632 ae3b1f
# it is ok to replace it by itself
454545 454545 019283
-------------------------------END OF EXTRACT----------------

You can get a list of options by running ./smooth -h
If you pass an unrecognized option, the program will stop.
Common options are:
-i inputimage
-o outputimage.bmp
-c configfile

Because of lack of support for other file formats using SDL_image, the output image has to be a BMP. Even if you put another extension at the end of the filename, the content will be BMP.

Plugins currently in development are:
[stream]: useful for roads, water streams etc where one chunk follows a line going in all possible directions across other chunks.
[smooth]: make the seam between 2 types of chunk smoother (like water land).

The source code can be downloaded at the following address: http://si-french.sf.net/smooth.tar.gz. The code is GPL so you know the rules.

Also, I do not guarantee the code will compile and run on your machine. It does on mine but YMMV. External libraries requried: SDL, SDL_image. The rest should be in your machine by default.

Please submit patches and plugins at: artaxerxes2 at iname.com.
I will only accept GPL or GPL compatible code.

Hope you enjoy it!

Artaxerxes

[artaxerxes]

Each plugin MUST have 2 special functions:

int plugin_parse(char *line)
char * plugin_apply(char colour[6])

line is a string holding the config file line given to the plugin to teach it to know what to do when asked to transform a colour into another. It returns < 0 when problem

plugin_apply takes a string made of 6 characters representing a hex colour (like 4f19bc), and it returns a string (6 characters) that represents the colour to replace with.

At first, I would say it doesn't matter what language it is written in, as long as it can be loaded with dlopen.

See plugins/ in the source to see an example.
A new page has been quickly thrown at: http://si-french.sf.net/tools/. Please use it for future access.

thx
Artaxerxes

[wjp]

What systems does it run on? Probably just Linux, right?

[artaxerxes]

It's been tested on linux only

Artaxerxes

[Ram dragon]

Hey, this is great. Smoothing land-ocean edges would be soooo useful.
I'm already thinking about how to implement an edge detection scheme, at least for land-ocean boundaries. Similar for land-river boundaries I suspect.
I had originally though to do this, but figured it'd be too complicated. But now I'm inspired.

One comment though, the links to download mockup

http://si-french.sourceforge.net/tools/mockup.tar.gz
http://si-french.sf.net/mockup.tar.gz

return a forbidden error.

[Ram dragon]

Actually, I don't mean edge detection since you can hard code the land-ocean boundary. I mean vectorizing the edge.

Ram Dragon

[artaxerxes]

Access to the files should be restored now.

Could you detail what you mean by "vectorizing the edge".

Anyways, more plugins will hopefully come this week. I'll also put a bit more documentation on how to write plugins. If anyone is interested to make this work on Windows, please feel free. The biggest part to do will be to modify the part for loading dynamic libraries on command and mapping a function. Linux uses dlopen() and dlsym() for that.

Artaxerxes

[Ram Dragon]

Well, vectorizing is too obscure a term. I meant that in order to apply the proper land-2-ocean chunk, you need to know the orientation of the edge in relation to the surrounding edges. Same would apply for doing rivers and mountain-2-(land/ocean) edges. I haven't really got into it yet, but I have a few ideas on how it can be done. Unfortunately, I won't be able to work on it until this weekend.

I'll post more detail after I've had a chance to work a few things out.

Ram dragon

[artaxerxes]

on a related note, I've completed about 90% of Stream, the new plugin to make roads and rivers get the proper chunks when changing direction.

2 last things need to be done for Stream: activating a check for 'trigger' and figuring out that blasting bug that makes 2 calls to the same function with the same parameters checking the same unchanged SDL_Surface return different values! Argh!

The check for trigger is to enable the change of chunk only if the slave shunk is beside a trigger chunk. For instance, say you have a 0.8 chunk-wide river, the remaining 0.2 is grass. You only want to change a river chunk into this chunk if it is beside a grass chunk.

Anyways, I'll let you know when it's all done. I've also written documentation on how to write a plugin. I'll post the new package soon.

Artaxerxes

[artaxerxes]

the new package is released and is available at http://si-french.sf.net/tools/.

The bug that I mentioned earlier has now been identified. In fact, the surface was changing as the apply()s went on. It's identified and it's already resolved.

For Ram Dragon and people interested in algorithmic:
I use a pretty neat trick for stream that you could maybe use for smooth (smooth is for the joining of land water for instance):
For stream, I created a table holding 16 values, each value being a colour. Then I simply applied a function that 'computes' a number based on what's around and I use this number as the index in the table.
Let me explain more in details:
I need to know where are the other stream chunks around the chunk I need to convert. Since I don't allow diagonals in streams, all I have to check is N, E, S, W. There are 16 possibilities:
0:
* none around
1:
* north
* east
* south
* west
2:
* ne
* ns
* nw
* es
* ew
* sw
3:
* nes
* new
* nsw
* esw
4:
* nesw

So I artificially placed a power factor to each cardinal direction. n=1, e=2, s=4, w=8, so that each combinaison results in a different value between 0 and 15 inclusive thanks to the function:
1*is_same(i,j-1,colour) + 2*is_same(i+1,j,colour) + 4*is_same(i,j+1,colour) + 8*is_same(i-1,j,colour)

is_same returns 1 if the colour at (i,j) is the same colour at what was given in arguments or return 0 otherwise or returns 0 if out of boundaries.
Then, all I have to do is to retrieve in my table the colour at index = result from the function.

Hope you might find that useful!

Artaxerxes

[artaxerxes]

Just to announce I've finished writing the second plugin "stream" which now will also check to see if a trigger chunk is beside a chunk to alter if trigger is not '*'.

It is also not recommanded to make the streams more than 1 pix-wide but it's up to you if you want to do it.

For a recap, stream convert a stream made of one colour into a stream made of different colours based on how the stream is turning around. All you have to do is to set mapping.txt (from mockup) accordingly to convert each colour to its proper chunk. You still saved tons of work and the result has no mistakes.

Included in smooth is a default map (rough.bmp) and defaults rules (smooth.conf) to convert a dark gray line into a multicolored line. Included in mockup is a config file (mappings.txt) to convert this multicolored line into the right chunks to make a pathway. You can test it, it works great.

As mentioned earlier, those programs compile and run on my machine but I cannot guarantee they will work on yours. It should compile on Linux ok, but not on Windows without making adjustments.

If you port the code to another platform, I'd be grateful if you could submit the patch so I can include it in other versions.

Thanks
Artaxerxes

ps: I'm gonna work on the "smooth" plugin now.

[Ram dragon]

Hey, very clever idea for the rivers. I've checked out forest-grass boundaries and they have the same 4 orientations. Should be easy to adapt river smoothing to forest smoothing.

Ram Dragon

[artaxerxes]

[smooth] (the plugin) is actually done at 70%. I don't think it will take long to have it finished.

Artaxerxes

[Skutarth]

This is all bloody awesome. THANK YOU!

[Skutarth]

Can you compile this for Windows (like last time) please? I don't have a compiler because I just had to reformat both of my comps.

[artaxerxes]

it is already compiled for windows thx to Colourless.
You can get the prog at: http://si-french.sf.net/tools/

Artaxerxes

[Skutarth]

Thank you, Colourless!

[artaxerxes]

A new plugin has just been created: smooth

This is the third plugin (after randomize and stream) to make the creation of maps for exult-based games so much easier.

smooth is used to make the transition between a type of chunk to another type of chunk in a smooth manner, using intermediary chunks.
For instance, the transition between land and water can now be similar to what Ultima 7 shows (using a beach type of chunk), done automatically.

smooth (the plugin) will take some parameters set in the config file and generate new colours wherever it was found an intermediary chunk should be used. It's now up to mockup to convert those colours into chunk numbers.

I haven't posted a screenshot yet, but it will come soon.

The 3 plugins are really the basis for making maps easily. As soon as this plugin will be ported to Windows, Win users will be able to create maps in a breeze. The source is already available and compiles on Linux.

To recap what the plugins are and what they do:
randomize:
*********
Converts a certain colour into another colour found in the config file. Each colour can be converted to a different chunk using mockup

stream:
******
adjust a colour forming a line (straight or curved) in an image, so that one colour only is used for each direction and combinaisons of directions. Useful for roads and water streams. mockup can convert each colour to the right chunk

smooth:
*******
modifies a blob of colour so that each possible type of border gets its own colour which can get converted to the right chunk using mockup

The source can be downloaded at http://si-french.sf.net/tools/

Thank you
Artaxerxes

[agentorangeguy]

Bump.

Does anyone know if this program still exists for download anywhere? I'd like to play around with it.

[Malignant Manor]

Smooth and Mock Up are in the Exult Source and Tools download but I have never used them. I think TFL used them to auto-generate their map.

[Dominus]

It's in exult's source /mapedit/tools/smooth

[agentorangeguy]

is there any documentation how to use it? i click on it but nothing happens. is it a command line thing?

[Malignant Manor]

Yes, you need to use the command line You can look at the files in these two directories if there is no documentation in the tools package (there should be).

[agentorangeguy]

The readme wasn't very clear on what to put in the command line. It says: "To call the prog:

./mockup
where contains a 8 bpp indexed image at 192x192 resolution (BMP, GIF and PNG)
and contains a simple colour u7chunk# association separated with a space."

so it would be (directory)\mockup c:\(image directory)\image.bmp then ??????

for , do I direct it to mapping.txt?

[Malignant Manor]

That should be right but you would likely need to manually edit which chunk will go to which color. You should try using the default map.png and the two mappings.txt files and look at the results in game to get a better feel for it. Then you can try with your own picture and mappings.txt settings.

Besides this, I'm not really sure how to use the other programs to try to improve the mapping.

[artaxerxes]

Hi,

I wrote the program, so I'll do my best to answer you.

mockup converts an image into a u7map. To convert it, it associates a pixel colour to a u7chunk and then creates a u7map "replacing" each pixel of your image by that u7chunk. "replacing" in quotes since it's not that simple, but you get the idea.

So, the program needs 2 things: an image and a file telling it that, for instance, black is chunk #123, white is chunk #163, etc. That file is called mapping and it is formatted like this:

COLOUR1 CHUNKNUMBER1
COLOUR2 CHUNKNUMBER2
...

COLOUR# is an RGB colour value in hex (FFFFFF is white for instance)
CHUNKNUMBER# is a u7chunk. There is a viewer in ExultStudio to know the number of the chunk you want.

With those 2 files (your image of a map and the conversion table or mapping), you call the program like this:

C:\>mockup.exe image.bmp mapping.txt

This will create a file "u7map" which you can use with Exult (or even U7 for that matter).

I hope this answered your question. Feel free to ask for any other clarifications.

Artaxerxes

[artaxerxes]

Besides this, I'm not really sure how to use the other programs to try to improve the mapping.

I'll try to answer that one. First with the plugin [random]. Suppose you want a forest in your map. If you picked a colour (say #0000FF) to represent the forest and you picked a chunk# of a forest, then every 0000FF pixel will be replaced with the same chunk which looks unnatural. The forest will look better with variety, but it's tricky to manually adjust every pixel. So with "smooth", you say that every occurrence of 0000FF gets randomly replaced with one of 0000FF, 0000FE, 0000FD etc. That will create a new image with randomized (preselected) colours. Now you add 0000FE and 0000FD, etc to your mapping.txt file and running mockup, you get a more natural forest.

The other plugins are [smooth]: this automatically detect edges and replace them with an intermediary colour, so you can transition between water and land or forest and land etc.

Finally [stream]. Imagine you want to make a paved road. There are several dedicated chunks for those (I am thinking of the highway in SI). If you simply used ONE colour chunk association, then if the road turns around, then goes up, left, down, right etc, you would be using the same chunk for each segment of the road, so it won't connect well as maybe the chunk you picked was showing the road going westeast. This plugin saves time by letting the computer analyze what you said is a road and automatically change the colour when the road veers in certain directions. Then all is left is to map those new colours to the correct chunk# and use mockup to create a final result.

Artaxerxes

[agentorangeguy]

Ok dumb question, but how do I figure out which color is what hex value? I have ms paint and Gimp but I can't figure out where to look or which setting to use to see which color is which hex value. I have a test map that I made that I'd like to try out but I can't figure out which is which. I used Paint to make the test image, just the basic 16 colors.

[Malignant Manor]

You can add a "Colormap" tab to Gimp's toolbox. If you use the eyedropper, it will highlight that color in the colormap and the "HTML notation" area at the bottom will have the hex number you need. Clicking on a color index also works if you can tell the colors apart easily by sight.

[agentorangeguy]

ok i ran the program but nothing is happening that I know of. both my image and mapping.txt are in the same folder (tools) as mockup.exe. Is there some sort of confirmation that the u7map file is made? where is it created at?

[artaxerxes]

You must run the program from the command line. So if your mockup.exe, image.bmp and mapping.txt are found in "c:\Documents and Settings\agentorange\games\exult\tools\", do the following:

Start->All Programs->Accessories->Command Prompt

This will get you a command line. Regardless of the default directory, do the following:

c:\whatever\>cd "c:\Documents and Settings\agentorange\games\exult\tools"
c:\Documents and Settings\agentorange\games\exult\tools>mockup.exe image.bmp mapping.txt

This will create a "u7map" file, which you can either:
1) use to replace the real u7map from your STATIC directory
2) place into a PATCH directory which exult can use if told to do so (see the documentation for Exult).

Have fun,
Artaxerxes

[agentorangeguy]

I'm sorry to be a bother, but I've done everything you've said, and I'm still not getting a u7map file anywhere. I even downloaded the latest exult tools here. There was no documentation in the tools folder. Are there any accompanying files associated with mockup.exe and smooth.exe? They are stand alone files in this tools folder, so I don't know if that has anything to do with it or not.

I even did a search for "u7map" on my comp to make sure it wasn't created in another folder, it wasn't. I don't know what I'm doing wrong.

[artaxerxes]

agentorangeguy: can you email me your map? I want to test it here.

My address is:
domain user = artaxerxes2
domain name = iname.com

Artaxerxes

[agentorangeguy]

Ok I sent it just now. Let me know if you get it. Th anks!

[artaxerxes]

ok, I've received and converted it successfully. What was the output you got when running the program? For me, I had:

>mockup agentorange.bmp agentorange.txt
The image file uses 5 colours
Colour map is successfully loaded from agentorange.txt
Done!
>

And in that directory, I have now a u7map file ready to be used (which I sent to you by email)

Artaxerxes

[agentorangeguy]

When I run the program from the command line, and do it exactly like you say, the program runs but nothing happens. Even if I type an incorrect command with mockup, it just goes to the command line. No output, nothing. I'm using the mockup from the exult tools. There must be something wrong with it, I don't know what else I could be doing wrong.

[paulo]

Sometimes cmd line tools can get confused about the location of arguments if you don't run them from the current dir (@!#$ gets complicated even on higher level languages)

run it from the command line on the same dir as the exe and with the arguments on the same dir (or with absolute path to arguments).

[artaxerxes]

I have just tried on a windows machine (XP Pro).

First off, the tool died because I didn't have SDL.dll around. That was easy to fix, but I did get a popup to tell me so, something that agentorangeguy didn't get.

Once SDL.dll acquired, I ran the command again:
C:>temp_folder>mockup.exe image.bmp mapping.txt
C:>

It returned with no message and did not produce the u7map as expected. However, it created a stderr.txt file. That file lists the problems the tool faced, in my case, an incompatible version of libPNG. So it seems that mockup.exe was linked with an ancient version of libPNG that newer libSDL.dll can't handle.

I was wondering if any Windows maintainer would like to create a new mockup.exe with the appropriate versions.

In the meantime, agentorangeguy, I will implement this alternative solution I mentioned in one email.

Thanks
Artaxerxes

[Dominus]

Hmm, libpng is not included in sdl, I think. Can you contact Kirben and point him to this thread?

[artaxerxes]

will do
thx Dom

[agentorangeguy]

I knew I wasn't crazy! Thanks guys, hopefully it won't be too hard to get it fixed.

[Kirben]

The tools package for Windows has been updated, to resolve that error.

mockup and smooth use the command line for output now, which should make their use easier too.

[Scythifuge]

If anyone wants to create a mockup map for Savage Empire, I would greatly appreciate that! It would give an idea as to the potential map size and possible modifications.

[artaxerxes]

Scythifuge:

there is a question of scaling to it match the scale of U7. For instance, how much of a chunk will be used for the dirtpath: the whole thing, half of it, or only a sliver. That will have an impact on the map. Having right now a quick glance at the map, I don't think you would be able to make it fit into 192x192. On the other hand, Exult supports multimaps I believe, so that could be used potentially.

Artaxerxes

[Scythifuge]

I'm thinking of just a comparison of the original Eodon map shape placed into Exult. The experiments I was doing involved using no chunks at all, but rather blocks to put the map together, similar to how Legos work. This is to compensate for a lack of z height in the original game. No matter the method, recreating Eodon will be a laborious task!

[agentorangeguy]

ok its actually doing something now but its giving me an error message saying colour 000000 not defined in mapping.txt. i'm not using the color in the image (pure black). What is going on?

[artaxerxes]

Actually you are. It's even the first colour defined in the image you sent me.

Here are the colour used in your image:
000000
00a2e8
b97a57
b5e61d
fff200

By the way, you _named_ it image.bmp but in reality it's an indexed PNG (but this is unrelated to your issue).

Artaxerxes

[agentorangeguy]

The problem i'm getting right now, is that it keeps saying certain colors are not defined. For example, it will say "colour 800000 is not defined" and i"m not even using that color. To work around that, I'll just define it as a random chunk and move on. When i run mockup agian, this time, it will list another random color and says i need to define it. It just keeps listing new colors it says my image has, even if I make an image with only one color.

I don't know what i'm doing wrong. Here is what I'm doing:

1. I make an image in the ms paint program. I'll save it as a .bmp file. I've tried 256, 16, etc.... so one question I have is: Which is preferred?

2. I take that image, open it up in GIMP, get the html notations for each color and attach it to a chunk in a mapping text file.

3. I index the file with GIMP, and I'll just click save.

4. I run mockup, it keeps prompting me to define a random colour. usually it will be like 800000 then 8080808 then 0080808 and so on... very weird.

Am I doing anything wrong in these steps? Am I saving the file wrong? I'm sorry to bother you all with this, I just don't know what is going on with it.

[artaxerxes]

I think I know what's happening. I've emailed Kirben about it.

The issue is with a potential buffer overflow. Kirben extended the buffer but it was left uninitialized. I've sent him a patch but he only knows how to prepare the windows binaries.

And to answer your question, a 256-colour BMP or (indexed) PNG is fine.

thx
Artaxerxes

[artaxerxes]

Agentorangeguy:

Kirben updated the Windows tools. Would you mind giving them a shot?

thx
Artaxerxes