Light

bmac2112 · Post by **bmac2112** » Tue Mar 04, 2008 5:33 pm

Are light sources already known to be quirky, or has this not yet been discussed (I can't find anything after searching the forums)?

Anyway, the behavior of light sources doesn't match how it works in the original U7. If I leave a candle on in a building at night and exit the building, Exult still maintains the reddish tint, while U7 transitions to a completely dark world. In Exult, if I'm anywhere near a building with a light source inside, everything in the outside world is lit up. This is especially noticeable at night in Britain with the sheer number of indoor lights.

There are also frequent occasions where the red tint never goes away. For example, in the middle of the night I was at Mack the farmer's house. I lit his candle. I left his house and walked west to the shore. The world was still tinted red. I went back indoors, turned of the candle, and everything correctly went dark.

After turning off his candle, I then entered the Britain moongate nearby. It took me somewhere (I'm ashamed it's been so long I can't immediately recognize the location, but it was right outside a dungeon/cave and had tropical trees nearby). Anyway, I entered the cave and explored for a while. I left, and outside it was getting lighter; it was somewhere in the hour of 5am. I used the Orb of the Moons to return to the Britain moongate, but everything was completely dark outside again.

Last but not least, THANK YOU to the Exult team. I've known about Exult for quite a number of years, and while I've tinkered with it in the past, this is the first time I've decided to play through U7 again, via Exult in particular.

MeddlingMonk · Post by **MeddlingMonk** » Tue Mar 04, 2008 6:28 pm

I've run into something like that, too, except what I've mostly noticed is sometimes, exiting a cave or dungeon and it's the middle of the day, and it stays reddish. But us just save and reload when that happens and the light is the way it should be. I think the devs are trying to make the lighting and weather more realistic (I was pretty damned started when I first saw fog) and I just put it down to teething troubles.

Post by **marzo** » Wed Mar 05, 2008 2:44 am

I think the devs are trying to make the lighting and weather more realistic

Try "more like it was in U7" and you'd be right. No, really -- try it in DOSBox and you'll see it: set time to 4AM and time rate to 5 to see dawn, for example.

But yeah, it is still clunky.

bmac2112 · Post by **bmac2112** » Wed Mar 05, 2008 1:29 pm

Well even the original U7 had some lighting bugs. For example, if you wait around at 4am, it gets lighter gradually, as you indicate. However, oftentimes in the original if you light a torch/lamp and turn if off during this transitional period, you lose the gradual transition. It "jumps" to a specific lighting level, then jumps from there to full daylight at the right time.

That's just something I noticed all those years ago when I played U7. I haven't tried to reproduce that lately, though. Hopefully Exult will avoid that situation.

The lighting was always was immersed me in to experience a lot for Ultima, ever since U5's day/night cycles. The quirky lighting in Exult kind of jerks me out of out the experience. I can't even begin to imagine how you would code it to "know" whether you're indoors or not... sounds tough.

Post by **drcode** » Wed Mar 05, 2008 2:37 pm

Well, we do know when we're indoors, since that's how we know not to display the roofs. So we have the potential to do the lighting better. It's just a matter of doing the work.

Post by **marzo** » Wed Mar 05, 2008 5:40 pm

So we have the potential to do the lighting better. It's just a matter of doing the work.

For one, I have noted that the originals seem to consider only light sources within a certain distance of the avatar (although I haven't measured the distance yet -- but something around 12-20 tiles sounds right), which is unlike what Exult does (count lights in each on-screen chunk); that might help. The same could be said of NPCs carrying lights -- they should light up the immediate area (which can be annoying as they move about...).

MeddlingMonk · Post by **MeddlingMonk** » Thu Mar 06, 2008 6:30 pm

BTW, speaking of lights and the original game...it's been a long time, but I seem to remember that in the original when you sleep in range of candles and they burn out overnight, NPCs (or at least companions) would replace them in the morning. And something not in the original nor in Exult...but is in the wonderfully awful U9...any chances of street lamps coming on automatically at night?

William · Post by **William** » Fri Mar 07, 2008 4:26 am

Yes, I also remember the original when the NPCs or party companions would replace burned out light sources when necessary, saying something along the lines of, "This needs replacing..." That was one of those little things that brought realism to the game and I really enjoyed that.

As for the street lamps, I remember in the original, and I think in Exult too, although I have not played for some time, the NPC in whatever building would turn on/off the lamps outside of their buildings as appropriate. This took care of many of the street lamps, but not all of them.

This has gotten me thinking of another thing I always enjoyed. The NPCs opening/closing shutters when appropriate. I haven't noticed this in Exult. Does it happen and I'm just missing it?

Post by **marzo** » Fri Mar 07, 2008 5:20 am

but I seem to remember that in the original when you sleep in range of candles and they burn out overnight, NPCs (or at least companions) would replace them in the morning.

Yes, it happened; Crysta had reported it quite a while ago and it is still in my to-do list of things to "fix".

The NPCs opening/closing shutters when appropriate.

It should happen in Exult too.

MeddlingMonk · Post by **MeddlingMonk** » Fri Mar 07, 2008 8:36 am

Well, yes, NPCs do turn on street lights. What I meant was all the street lights going on by themselves at dusk and then off again by themselves at dawn. In a way, for citizens to light the lamps themselves--and if there's no one nearby, the lamp doesn't get lit--isn't very realistic. In a place with gas or oil street lights, there would be people whose job it would be to go around lighting them all at dusk and dousing them at dawn. Not that Exult needs a special army of NPCs to turn the street lights on and off. It could just be triggered by time.

bmac2112 · Post by **bmac2112** » Fri Mar 07, 2008 2:11 pm

I love the fact that NPCs turn on street lamps. On one level, I supposed it's somewhat unrealistic that you can enter a "major" intersection and there are no lights, but the having every single light on everywhere can be just as unrealistic, not to mention the fact there are no people ever wandering the streets to warrant having lights on in the first place. But NPCs doing the lighting adds so much there's a total "net gain" in immersion, IMO.

As for shutters, yes, I see NPCs opening/closing them often in Exult, just like U7.

bmac2112 · Post by **bmac2112** » Mon Jul 20, 2009 11:02 am

Has anyone given further thought to making the lighting work more like the original?

I took a look at the code and I had a difficult time understanding this part (because I'm not a programmer, not because of the code), but when Exult is counting light sources in chunks, can it simply ignore lights in indoor chunks if you're currently outdoors?

bmac2112 · Post by **bmac2112** » Thu Jul 23, 2009 5:11 pm

So I did some work privately and changed the way the renderer counts light sources.

- If the main actor is indoors, it counts all lights
- If the main actor is outdoors, it counts only outdoor lights
- If the main actor is in a dungeon, this was left as-is

Also, instead of counting every light source in every loaded chunk, I tried a few things:
- Based on distance (20 tiles)
- Based on the light being within boundaries of the view window or not

Next I carefully examined how the original U7 did it. Different light sources have different radii at different light levels. I started by mapping out lamp posts.

http://img199.imageshack.us/img199/9972 ... mppost.gif
(Sorry, I don't know how to directly link or embed)

Each block in my image represents a 8x8 tile in U7. The diagonal line in the center represents the lamp post. The white line is the boundary where the bright light/palette is triggered, and the gray line is the boundary where the softer light/palette is triggered. Incidentally, candles seem to have a) only the softer palette and b) a smaller area, though the pattern is similar (I haven't mapped it out yet).

At first, when I mapped the bright area, it seemed to be a clear isometric square clipped at the right and bottom edges. Simple enough. But the softer edge is quite bizarre. It seems like it's getting clipped on the right and bottom somewhat, but at the very far right-bottom, it has a squiggly shape.

- I can't imagine an elegant algorithm to account for this. This seems to have a human touch.
- Chances are U7 didn't compute the area every single time. Too much work.
- Chances are they used a static bitmap mask (like a lightmap) for each kind of light source.

Do any of you Exult programmers recall seeing a bitmap array anywhere that is reminiscent of the pattern in my image? Do you know if U7 used static masks for lighting like this?

Post by **Malignant Manor** » Thu Jul 23, 2009 6:12 pm

For one, Exult leaves out at least one palette change for lights. One candle will get one palette change, 4 will get another brighter version, and 10+ gets yet a brighter version. The 5 candle shape, torches, and the lamp post seem to act as 5 light sources.

2 light sources in an area extend the range slightly even when not changing the light level.

Post by **marzo** » Thu Jul 23, 2009 7:20 pm

At the points were the isometric square was clipped -- did, by any chance, the lamp post go offscreen when you were mapping the lit area at those points?

Post by **Malignant Manor** » Thu Jul 23, 2009 7:51 pm

I know that from my testing, the N, S, E, and W directions extended slightly off screen. The diagonals were very wrong and only went a maybe half that as if calculating both the x and the y instead of a radial effect.

bmac2112 · Post by **bmac2112** » Thu Jul 23, 2009 8:16 pm

Marzo, you're exactly correct. The base of the lamp went of screen.

Moving toward the left, the pallete when from bright to soft similarly when the base was off screen, then completely dark when the top of the lamp was offscreen.

However, moving stright north of the lamp had no discernable pattern. The entire lamp and circle of light are off the screen for several tiles before the palette goes completely dark.

The softer palette didn't seem to correspond to anything I could determine. I thought it might have something to do with the cirlce of light the lamp casts, but that wasn't it.

Moving diagonally, it becomes apparent that distance is also a factor, because different parts of the lamp are still on screen, but the palettes don't switch the same way I do when I move strictly N/S and E/W.

It was some of this odd behavior that prompted me to map it, so I could visualize what was going on and try to infer an algorithm.

Post by **marzo** » Thu Jul 23, 2009 9:04 pm

It seems that the variables so far are:
* light sources seem to have different illumination power (as Malignant Manor discovered, it seems that multiple light sources seem to act as a single more powerful light source);
* distance from the light source seems to be a factor (apparently in an isometric-square form);
* the number of light sources present (I know enough of them affect the palette, but as Malignant Manor discovered, they also affect the range);
* whether the light source is onscreen or not (but there is some leeway, particularly to the left and to the north).

I have a few questions (which I can't test myself now); I suggest using the weakest possible light source since the strength seem to affect range. To make it standard, take a candle (if memory serves, it is frame zero of the lit light source shape); I guess it would qualify as the weakest light source, considering that there are candelabra, campfires, torches and lampposts.
* How far north do you have to walk to trigger the palette changes?
* How far west do you have to walk to trigger the palette changes?
* How far to the north-west (diagonal walking) do you have to walk to trigger the palette changes?
* Is the north-to-west diagonal at a 45-degree angle (that is, does walking 1 tile to the southwest or to northeast preserve lighting condition)? Or is it at a different angle (say, walking 2 tiles west and 3 tiles south)? Or it isn't even a straight line?
* How far (diagonally) from the light source between the light changes?
* Does distance along the z axis affect lighting condition or is it only distance on the x-y plane important?
* Repeat all of the above questions with two candles at the same location.

Post by **Malignant Manor** » Thu Jul 23, 2009 9:19 pm

Here's a few screenshots I took earlier. I will upload a kind of example text document of light format following U7 but allowing for any light source number instead of 1 and 5.

Post by **Malignant Manor** » Thu Jul 23, 2009 9:27 pm

Oops, second picture, 1st row, second column should say 5 light sources.

Here is the tracker with some sample light source information for both games.

bmac2112 · Post by **bmac2112** » Thu Jul 23, 2009 11:31 pm

I'm definitely interested in gathering more info, but I'm going to be offline for a few days. I'm out of state and should have installed U7 on my laptop. :/

Post by **Malignant Manor** » Fri Jul 24, 2009 1:08 am

Z height didn't seem to matter (but should likely be considered for Exult). Sizes below can vary from where the origin is considered of the Avatar and the candle. NW and NE are in the picture but I'll leave that for how you want to figure out the length.

Two candles:
~190 pixels W
~170 pixels E
~160 pixels N
~120 pixels S
~100 - ~110 pixels SW
~100 - ~110 pixels SE

http://i156.photobucket.com/albums/t13/ ... andles.png

(You may want to edit my two previous photos so that they aren't hot linked since they have a large file size.)

Post by **Malignant Manor** » Fri Jul 24, 2009 1:14 am

Oh, and when there are two Avatars, the one furthest away from the light source is the correct one. The diagonal line in the NW and NE pictures might be confusing otherwise since they coincidentally coincide with the screenshot that had the light sources in view.

Post by **Malignant Manor** » Fri Jul 24, 2009 1:37 am

Can you do lighting different than U7 and have say a different radius for each light strength increment (1, 5, 10)? Say one radius for light sources 1-4 and if you go outside the radius of those lights it is dark. Light source strength >= 5 && <10 would switch to the 1-4 light source palette once a certain range was passed. With light source strength greater than 10, you would have 3 palettes to go through before finally going dark. I say just pick a radius you feel comfortable with for 1-4 str and wing it for the rest.

Post by **marzo** » Fri Jul 24, 2009 9:49 pm

You know, that (duplicating functionality without bothering to completely reverse engineer the original) is the best suggestion ever.

William · Post by **William** » Sat Jul 25, 2009 2:40 am

I LOVE you guys for taking a look at the light issues. It was always one of the things that made the original so engrossing. As with many of the feature's you've worked on, you'll probably end up making this better than the original too!

bmac2112 · Post by **bmac2112** » Sat Jul 25, 2009 11:32 am

Agreed - not only is reverse engineering this thing is a bit of a pain, but we'd be getting to a point of diminishing returns, especially since U7's lighting doesn't always make sense.

The distance should have something to do the size and ratio of the screen, IMO. When I set a distance of 20 in my experiments, it was really nice walking north to south. But walking east to west, the lamp was still clearly onscreen, and it looked weird while the world appeared dark. So perhaps light areas should be oval shapes, since U7's viewport is wider than it is tall.

BTW, on a somewhat related note, I had to force Exult to dirty the whole screen every cycle when playing with different lighting. When I was monitoring light effects within Exult, I noticed it didn't always change the palette after the chunk was out of range. There wasn't a clear border where a light palette would be changed back and forth.

For example, if I was walking slowly, when I hit the first tile where the light palette would be utilized (i.e. in range of a lighted chunk), I would then take one step back, but the palette didn't change back. It didn't seem to have a clear boundary where it would change back - rather random.

But, if after that single step back I opened inventory, the palette would correctly change back to dark. The gump renderer obviously triggers whatever it is that makes Exult check for the palette change. I found that by cheating and always making the whole screen dirty (set_all_dirty or something like that), the palettes would change consistently.

Andrea B Previtera · Post by **Andrea B Previtera** » Sun Jul 26, 2009 4:50 pm

Would it be correct to assume that lighting was substantially buggy if not broken in the original too? I clearly remember being puzzled again and again by how the light sources influenced the world even back in 1992.

In that case, is it project-coherent to replicate the original behaviour even if proven to be flaky, or fix/enhance it?

Post by **Dominus** » Sun Jul 26, 2009 7:27 pm

In that case, is it project-coherent to replicate the original behaviour even if proven to be flaky, or fix/enhance it?

good question and hard to answer. We had some complaints over and over again, that we do not replicate the original good enough, especially the exploitable bugs...
IMO I would add a more natural lighting but who knows what others think is best

bmac2112 · Post by **bmac2112** » Mon Jul 27, 2009 1:14 pm

Malignant Manor, how did you determine the # of lights affects the overall range of each? I'm unable to see that.

Post by **Malignant Manor** » Mon Jul 27, 2009 4:17 pm

Malignant Manor, how did you determine the # of lights affects the overall range of each? I'm unable to see that.

It only seems to work diagonally, which may be why you can't reproduce it. I tested with 2 candles and since the candles were still on screen, you should be able to turn the furthest candle away off in the furthest area there is still light and watch it go black. You could also walk the furthest one candle will go and then light the another one near next to the previous candle (but farther away from you). Adding a third candle seems to add very slightly to the distance. I never really messed with it more.

Andrea B Previtera · Post by **Andrea B Previtera** » Mon Jul 27, 2009 5:36 pm

@Dominus

Maybe Exult is to the point (and probably go there by the release of version 1.0) where it must decide whether it should aim to perfectly replicate the original experience, or just allow the user to play the games in the smoothest way (thus, enhancing the experience where possible)

Given that DosBox and/or virtualized Dos can completely fullfill the first goal ,something that wasn't possible until a couple years ago, maybe Exult could move onward and full steam to goal #2.

bmac2112 · Post by **bmac2112** » Mon Jul 27, 2009 10:21 pm

Marzo,

I'm trying to figure out how U7 determines how strong a light source is (e.g. candle vs. lamp post). It doesn't seem to be in TFA.DAT.

TFA.DAT - Candle (top) and Lamp Post (bottom)
0100 1000 0000 0010 0100 0000
1110 1000 0000 0100 0100 0000

I tinkered with each flag that's different, and none seem to be it. Are there other files that record shape/item attributes? It wouldn't be ideal to hard code it, but there may be no choice...

bmac2112 · Post by **bmac2112** » Mon Jul 27, 2009 10:37 pm

Marzo,

I'm trying to figure out how U7 determines how strong a light source is (e.g. candle vs. lamp post). It doesn't seem to be in TFA.DAT.

TFA.DAT - Candle (top) and Lamp Post (bottom)
0100 1000 0000 0010 0100 0000
1110 1000 0000 0100 0100 0000

I tinkered with each flag that's different, and none seem to be it. Are there other files that record shape/item attributes? It wouldn't be ideal to hard code it, but there may be no choice...

Never mind, it's in Usecode.

bmac2112 · Post by **bmac2112** » Mon Jul 27, 2009 10:48 pm

Quote:

Marzo,

I'm trying to figure out how U7 determines how strong a light source is (e.g. candle vs. lamp post). It doesn't seem to be in TFA.DAT.

TFA.DAT - Candle (top) and Lamp Post (bottom)
0100 1000 0000 0010 0100 0000
1110 1000 0000 0100 0100 0000

I tinkered with each flag that's different, and none seem to be it. Are there other files that record shape/item attributes? It wouldn't be ideal to hard code it, but there may be no choice...

Never mind, it's in Usecode.

Never mind again. it's not in Usecode. It must be hard coded in u7's original renderer.

Post by **marzo** » Tue Jul 28, 2009 6:57 pm

It doesn't seem to be in TFA.DAT.

Never mind, it's in Usecode.

Never mind again. it's not in Usecode.

If you had asked, I could have said it was hard-coded: TFA is fully known (hence it could not be there) and usecode could not be as it is too slow for this task (at least in the originals it would be, as U7 already pushed the hardware of its day to the limits). Likewise, the other data files are either not appropriate or fully known.

It wouldn't be ideal to hard code it, but there may be no choice...

Hard-code it? Me? After all the hard-work I have put into *de*-hard-coding stuff? Never!

Seriously, even Malignant's Manor findings about relative strength (which he posted to the tracker) are already in a text-msg-format which is heavily used by Exult/ES to store data that the original hard-coded. There is still a lot to go, but bit by bit we are getting rid of all hard-coded data.

Post by **Malignant Manor** » Tue Jul 28, 2009 7:27 pm

Malignant's Manor findings about relative strength (which he posted to the tracker)

Well, I just winged it for many shapes and haven't tested/left up to you many choices. I just tried to get a quick list of all things that might need it or people might alternately want it in a format similar to what is used for other de-hardcoded data. (I think the lightning whip is in SI data but not BG and am unsure atm how it worked in either.) I think I also forgot the Light Helm in SI.

Do you want me to go through and test strengths? (Most shapes would be tested in SI since I don't know how to input the alt-255 or whatever cheat in BG.) I'm currently working on SI style paper doll templates for dev games and multi-racial/sex avatars but could take a break to do this.

Post by **marzo** » Tue Jul 28, 2009 7:36 pm

Do you want me to go through and test strengths?

Thanks for the offer, but it won't be needed (since I am already doing that).

Post by **Malignant Manor** » Tue Jul 28, 2009 7:40 pm

I'm sure many won't make sense though if the original only used either 1 str or 5. (Think max sized fire field vs torch. Both likely give the same lighting. Some things probably don't give off light that should like the bunson (sp) burner) Will you stick to originals or add more appropriate strengths?

Post by **marzo** » Tue Jul 28, 2009 8:02 pm

The originals seem to use a bit more than that (1 or 5); using candles as a reference, I have encountered objects that seem to be equivalent to 2, 3, 5, 6 and 7 candles (all candles being placed in the same (x, y) coordinates). For example: the firepit frames seem to be, in order, 0, 2, 4 and 6 candles (that amounts to twice the frame; hacking in more frames result in the same progression), while lit lamp posts and campfires (shape 825) seem to be equal to 7 candles.

Post by **Malignant Manor** » Tue Jul 28, 2009 8:53 pm

It's good (and a bit surprising for an old game) to know the original didn't use shortcuts on the lights even if the way they rendered it seems screwy.