Pretty-printed the format; there's so many "0 padding bytes" that I think it's clear these are 16-bit fields. Makes the formats a bit easier to read, too.. =)

Crime files
-----------

Header (4 bytes)

ofs | datatype | description
----+----------+------------
0 | 16 bits | Number of participants
2 | 16 bits | Number of events

Followed by participant records

Participants (48 bytes)

ofs | datatype | description
----+----------+------------
0 | 16 bits | always 0xffff
2 | 16 bits | ?
4 | asciiz | 32 bytes for participant role + zero byte
36 | 16 bits | ?
38 | byte | Mastermind flag. 1 for mastermind, 0 for normal
39 | byte | ?
40 | 16 bits | always 0
42 | 16 bits | ?
44 | 16 bits | Rank of Participant, used solely for scoring
46 | 16 bits | ?

Followed by event records

Events (44 bytes)

ofs | datatype | description
----+----------+------------
0 | 16 bits | Person doing the event (so person sending a message or committing a crime)
2 | 16 bits | Always 0
4 | 16 bits | Message ID
6 | asciiz | 32 bytes for description of event + zero byte
38 | byte | Person the event is happening to (00 for crimes)
39 | byte | Type of event.
40 | byte | Bit-mask of items received by the person doing the event.
41 | byte | Bit-mask of items destroyed by the event.
42 | 16 bits | Score of event.

Note that last event is always just a special marker (empty event) starting with 0xff as the first field, otherwise zero.

Followed by objects until end of file

Objects (18 bytes)

ofs | datatype | description
----+----------+------------
0 | asciiz | 16 bytes for name of object + zero byte
17 | byte | Picture ID of object
18 | byte | Always 0xff

Event types are:
id | name
---+-----
02 | Sent message
03 | Received message
08 | Met with
09 | Was met by
20 | Crime

02/03 and 08/09 must happen in pairs

Picture IDs are:
id | name
---+-----
00 | Processed Drugs
01 | Sniper Rifle
01 | Weapons
02 | Raw Drugs
03 | Chemicals
03 | Supplies
04 | Explosives
05 | Alarm Bypass
05 | Product
06 | Photographs
07 | Hostage
07 | Pilot
07 | Escapee
08 | Bomb
09 | Payoff
09 | $1,000,000
09 | Gold Coin
0a | Photographs
0b | Camera
0c | Blueprints
0c | Mission Plans
0d | Stolen Goods
0d | Jeep
0d | Sample Item
0d | Forgery
0d | Equipment
0e | Passwords

Breakthrough: http://dl.dropboxusercontent.com/u/75238938/ca_pic2.png

...let's just say that there's more than one way to store a bit stream.. and naturally I tried the wrong one.

PIC format (format flag 0xF)

ofs | datatype | description
----+----------+------------
0 | 16 bits | Format flag.
2 | 16 bits | Width, always 320
4 | 16 bits | Height, always 200
6 | 16 bytes | Pixel color mappings for cga mode (apparently)
22 | byte | Max LZW dictionary bit width, always 0xB
23 | LZW data | image data in LZW+RLE compressed format

PIC format (format flag 0x7)

ofs | datatype | description
----+----------+------------
0 | 16 bits | Format flag.
2 | 16 bits | Width, always 320
4 | 16 bits | Height, always 200
6 | byte | Max LZW dictionary bit width, always 0x0B
7 | LZW data | image data in LZW+RLE compressed format

The image data is stored as LZW compressed RLE stream. The LZW resets when the dictionary gets full (i.e, there's no separate reset signal).
Under the LZW the data is compressed with RLE, so that if a pixel byte is 0x90, the previous pixel is repeated as many times as the next byte says; if the repeat value is 0, the pixel value is 0x90.

To reiterate, the RLE works this way:
aa 90 bb
if bb = 0, output is "aa 90"
if bb != 0, output is "aa" * (bb+1)

And yes, if you want a stream of 90's, you do 90 00 90 xx.

Each pixel byte represents two 16-color pixels. I'm not sure what they're doing with palette, it's
possible they're just using the "normal" 16 color palette.

Okay, the palette is wrong; http://dl.dropboxusercontent.com/u/75238938/ca_pic3.png - either it's (still) somewhere in those pic files, or it's hardcoded in the executable(s). In any case, the MPS labs logo at the start uses the "standard" 16 color palette.

Here's win32 binaries for a quick image viewer; http://dl.dropboxusercontent.com/u/75238938/dispic_v0.zip.. Give filename as parameter and it'll do its best. Note that only 0x0F format images are supported in that quick version. I'll improve it and post the sources once I've cleaned them up..

Excellent work!

On palette..

MGRAPHIC.EXE seems to contain some data that looks like vga palette data (around offset 0x14d2 onwards) - triplets of bytes with values below 64.

I figured that the game probably doesn't change palette all too much, so I might as well capture the palette once and be done with it.. unfortunately, at least the gender selection is just the standard palette with color #5 changed to black. This, obviously, isn't the palette used in-game. Sigh.

edit: actually, trying this with a couple of images looks like that's actually.. it.

Okay, pic viewer source and all format data gathered so far can be found at: https://github.com/jarikomppa/covert_action

Wrote a quick .CAT unpacker and a pic2png utility. Can be found at github. Also posted win32 binaries.

One thing that should be checked is that whether the game cares about the 'magic' value in the catalog files. Later games from microprose also use .cat files, but that magic field is gone (it may be checksum, it may be int handle, int may be some weird flags, who knows..)

edit:...aaand no, the game doesn't seem to care that I changed all the 'magic' values in cities.cat to garbage.

Great -- I've got a tonne of things happening this week, but hopefully soon I'll continue on the crime decoding. I'm sure I'm close to figuring out what the mystery fields in the participants mean, and should be close to getting the text files to work with new crimes (albeit overwriting old crimes).

Still having a pain getting debugging software to work in virtual machines though, so finding where the mission sets are will take a while longer.

Added a .cat file builder..

Ha, covert action loaded my pessimally stored pic file \o/

http://dl.dropboxusercontent.com/u/75238938/ca_pic4.png

I figured the first sane step in encoding pic files would be to store them pretty uncompressed, e.g, RLE spans are not detected (just the 0x90 code is encoded as 0x90 0x00) and LZW just stores single-symbol codes. The resulting file is over 40k (while it should be closer to 8k), but it loads!

Haven't posted the png2pic yet, maybe later in the evening..

EDIT: since I used the 0x07 format of the PICs, even the CGA mode kinda works: http://dl.dropboxusercontent.com/u/75238938/ca_pic5.png

Added RLE compression (still with dummy LZW) and that alone dropped the file size to around 10k, which I find reasonable. The rest is pretty much optimization..

So yes, if any GOGgers are reading, it's possible to replace the quit ad screen with custom graphics - gog ad, maybe? =) (not sure if that's in line with your contracts, though)

The png2pic tool now posted on github.

On PAN files.. comparing the files and looking for similarities, I've found out the following. For almost every thing there seems to be an exception..

ofs | datatype | description
----+----------+------------
0 | 32 bit | header tag 'PANI'
4 | byte | always 03
5 | byte | always 01
6 | byte | always 01
7 | byte | always 00
8 | byte | always 03
9 | 15 bytes | color mapping table from color 1 onwards
| | (color 5 maps to color 0)
24 | 5 bytes | always 0
29 | 16 bit | width -1
31 | 16 bit | height -1
33 | 16 bit | ? 1, 3, 4, 5
35 | byte | ? 0, 1, 2,
36 | 16 bit | ? 0, 1, 7, 5
38 | 16 bit | ? frame count?
40 | 16 bit | ? often 0xc8
42 | 8 bit | ? often 0x0b

In 18 of the animation files, byte at offset 42 is 0xb, followed by a variety of different kind of data, hinting that the animation frames might be delta-encoded PIC data frames.

When offset 42 is 0xb, offset 35 is always 1 and offset 36 is always 7, so these may be format flags. For all other files in this category, offset 33 is always 1, except for bustout.pan, for which it is 3.

If offset 40 is 0xc8, offset 42 is always 0xb. binos3.pan has 0xb at offset 42, but offset 40 isn't 0xc8.

For files where offset 42 isn't 0xb, a 0xb byte can be found later on (offset 0x220 for blding*.pan and title2.pan, but
0x21e for credits.pan).

For the curious, this is how I'm comparing binary files for common data (who said photoshop is only for artists?) http://dl.dropboxusercontent.com/u/75238938/ca_pic8.png

Added a simple binary patcher to the github.. here's a simple binary patch that attempts to fix the "stassi" typo:

world1.dta 918 105
world1.dta 919 0
world0.dta 846 105
world0.dta 847 0
final.exe 70636 105
final.exe 70637 0

..that doesn't seem to affect ongoing games as the organization names come from the save file. Use at your own risk, take backups and all that.