Spent a while looking at the files, figured this could be used as a starting point for other people if they're interested..

Files listed as [bytes] [filename]

2654 README.BAT
1437 BINSTALL.BAT
1437 INSTALL.BAT
27 COVERT.BAT

Original game batch files for installing (and installing from B drive), launching the game and reading the readme..

9360 TSOUND.CA
6331 ISOUND.CA
672 NSOUND.CA
18207 RSOUND.CA
16333 ASOUND.CA

Sound drivers for "tandy", "ibm", "nosound", "mt-32", "adlib". Looks like x86 code, header starts with MZ (as in, dos executable). Likely loaded as overlays.

56449 CITIES.CAT
56739 FINAL3.CAT
45473 FINAL4.CAT

CAT files appear to be file libraries (kinda like doom wad, or zip), each starting off with a dictionary followed by the actual data.

The first 2 bytes is the number of records in the file, followed by a 24 byte record.

First 12 bytes of the record is the filename (8.3, padded by zero bytes). The rest appears to be 2+2+4+4 or 4+4+4 bytes, which has to include the offset for the file, and likely contains the length of the file. As for the rest, no idea (as of yet).

012345678901234567891234
filename.extAABBCCCCDDDD

DDDD appears to be the offset.
CCCC appears to be the length.
No idea what AA, BB or AABB means; possibly checksum.

CITIES.CAT contains the city pictures. FINAL3.CAT and FINAL4.CAT contain pictures for the end-of-mission holiday scenes (laundromat, beach, etc)

2756 FONTS.CV

Assuming this contains the fonts..

1968 README.DOC

The same text that is printed by README.BAT; basically release notes.

1124 WORLD0.DTA
1124 WORLD1.DTA
1124 WORLD2.DTA

City data for the three different world maps used in the game. Fixed-length records again; the three files are the same length, and start with the two 2-byte values 0x0010 0x0010.

One city record appears to be 34 bytes long. After the 16 cities, data changes to 16 organizations with a 36 byte record size.

2448 PROSE.DTA

Some of the strings used in the middle of the game, like what your CIA contact says or your boss congratulating
you for arrest, etc.

1424 CRIME0.DTA
1108 CRIME1.DTA
1560 CRIME2.DTA
1736 CRIME3.DTA
1204 CRIME4.DTA
1468 CRIME5.DTA
1336 CRIME6.DTA
1556 CRIME7.DTA
1780 CRIME8.DTA
1468 CRIME9.DTA
1380 CRIME10.DTA
1288 CRIME11.DTA
1332 CRIME12.DTA

Data for the 12 different types of crimes. The format is similar as we've seen before; leading with two 2-byte values for number of certain records, then fixed-length records of those amounts.

First records are for the participants (48 bytes each), followed by mission events? (44 bytes). The result doesn't
match the file size, apparently there's additional records related to the items in the plot (18 bytes each).

There's doesn't seem to be a 2-byte counter for the items at the beginning of the file; maybe the items were a late addition?

23357 TEXT.DTA

Lots of the game strings. Messages, bulletins, etc.

10390 EGRAPHIC.EXE
6756 MGRAPHIC.EXE
9428 CGRAPHIC.EXE
8852 TGRAPHIC.EXE

Graphics drivers, probably loaded as overlays. (MGRAPHIC looks like the "native" as it requires least code).
C)ga, T)andy, E)ga - which leaves M)cga/vga.

7325 INTRO.EXE

Intro sequence?

51575 BUG.EXE

electronics minigame

20247 CODE.EXE

code-breaking minigame

80431 TAC.EXE

break-in minigame

29813 CHASE.EXE

car-chase minigame

7761 COVERT.EXE
980 MISC.EXE
83589 GAME.EXE
80589 FINAL.EXE

Different bits of the game. No idea what the misc.exe is for, covert.exe is probably just the launcher, and game/final cover the rest of the game.

14560 BUSTOUT.PAN
11102 RESEARCH.PAN
8838 WARNING.PAN
8232 ESCAPE.PAN
15840 BUGCAR.PAN
16268 SURNDR01.PAN
14924 OFFICEM.PAN
9952 CRYPTO.PAN
18816 BRIEFING.PAN
19418 BLDING04.PAN
7718 FOLLOWED.PAN
15750 BLDING03.PAN
7476 CAPTRD01.PAN
15030 OFFICEF.PAN
16364 BLDING02.PAN
9846 TITLE2.PAN
11180 COMM.PAN
7866 HQ.PAN
16920 BLDING01.PAN
16680 INJAIL.PAN
12228 BINOS3.PAN
11402 INTRGATE.PAN
16962 CREDITS.PAN
21710 WOUNDED.PAN

PAN files are the graphics for all of the animated sequences, starting with a PANI tag which suggests they're related to some tool not specific for this game (as game data files are tagless).

All of the files appear to contain the same first 28 bytes, most likely containing stuff like resolution. The data itself
is very likely to be compressed.

The format doesn't seem to be FLI/FLC (autodesk) or ANM (deluxe paint animator) which were popular at the time..
however, some sort of RLE scheme is likely to be in use, so that only the changed parts of the image are stored. Note how the WOUNDED animation is the largest file; the animation changes a lot of screen pixels every frame.

2263 LABS.PIC
8965 AD.PIC
11252 HOTEL.PIC
7871 GUYS3.PIC
7816 GUYS2.PIC
2568 GENDER.PIC
6688 FACESF.PIC
4222 AFRICA.PIC
6106 FACES.PIC
5896 EUROPE.PIC
6594 EQUIP2.PIC
6136 EQUIP1M.PIC
6266 EQUIP1.PIC
5062 SNEAKIN.PIC
10339 SPRITES.PIC
10370 SPRITESF.PIC
10536 STREET.PIC
3128 BOARD.PIC
4148 WIRETAP.PIC
9995 CHASE.PIC
6552 CENTRAL.PIC
4207 CARS.PIC
5793 TRAINING.PIC
2043 CAMERA.PIC
2695 BUGS.PIC
917 ICONS.PIC

The PIC files are (surprisingly) pictures, which seem to be compressed. Every file starts with a 6-byte header:

AABBCC

AA - either 0x7 or 0xF, possibly format?
BB - always 320 (horiz. resolution)
CC - always 200 (vert. resolution)

The compression scheme is likely to be some rle- or lz-based one given the age of the game (zip was "heavy" back then).

I hunted down picpac.exe that microprose released along with another game of theirs, and that uses LZW for compression. The files generated by picpac don't seem to be 1:1 with the ones in SMCA though, the header is slightly different (has more bytes before the width/height fields)

Compressing the gender selection picture (mobygames->photoshop->pcx) gives similar, but not exactly the same file sizes (~2k without options, ~3k with -8 option, 2.4k with -w2, gender.pic being ~2.5k), which suggests that
the compression might be an lzw variant.

10936 CV0.SVE

Save file(s), likely a dump of the whole game state, including your career data.

28919 PLOT.TXT

Strings for the boss (de)briefings.

6614 CLUES.TXT

Strings for the clues, like "xyz has been traced to a known blackmailer."

I'll send my findings later today. I've managed to almost make my own crime, with only needing to sort out the clue text.

Tools to turn the missions into editable text files and back should be relatively easy to do (once we figure out the fields, I mean).

Personally, I'd love to get those graphics files open, but that'll take some doing..

Oh incredibly easy -- I've already started doing a quick python script to do so with the crime files. The big, big thing that would be needed for making it usable is a crime checker, which makes sure the crime events can actually happen, since there's some logic as to how it works. I'll explain later tonight when I upload how the crime file works, along with the logic I can grab from it.

Getting the graphics files would be awesome - I also would love to get a proper dos debugger on this, so I can figure out where it's holding the mission sets data (doesn't seem to be in a seperate file) and then be able to either change or add mission sets so we can add the new crimes instead of purely overwriting what's already there.

On exciting news, it appears you don't need a mastermind involved in a crime; which could make some interesting new crimes!

----------------

Quick question (as I don't have the GOG version -- just my own floppy version):

Does it have a file called FAME.DTA ?

If not, then I'll upload a blank version, since this is needed for high scores to work (and a lot of versions of this out there doesn't have it)

I've listed all the files in the GOG version above, so, no. Please do write up the stuff you've found for others to play with =) OpenWatcom, maybe? I'm not sure if it still has the DOS stuff in it, though..

If it's impossible to make it see more than the 12 crime files, some kind of crime file manager thingy could be doable, although that could mess up save games..

Here's my one then - https://dl.dropboxusercontent.com/u/101876491/FAME.DTA

I'll send a message to the mods to see if they can add it to their download. Alternatively, you can just create a blank file called FAME.DTA, and the game will set it up when you set a high score. I will do ;) Nothing's impossible, though we'll see how easy it'll be. The engine does seem pretty resiliant though, which is great. Also lots of debug messages in the game when things goes wrong.

The big problem is really finding a debugger which works with dosbox. There is a build of dosbox with a debugger built in, but it won't break when a given memory address is accessed, which is what I need. SoftICE used to be great in the day; but apparently it won't work with dosbox. Makes me wish I had a dos machine hanging around here.

Soo.. virtualbox with freedos?

The format is also different from civilization (1991) pic file format, and railroad tycoon deluxe pic format.. It looks like they didn't use this exact file format for long, but right after this figured out a good format and then used that (or pretty similar) for a long time, because civedit and darklands and several others use nearly, if not exactly, the same format.

I also dug up f-19 demo and it has .pic files, but those look different again.

Oh well.

I dumped data off one image as 1,2,3,4...16 bit values at 0..15 byte offset, and it all looks gibberish. The first 19 pixels of the image are black, so there should be a bunch of pretty small numbers at start, like with rle you might have 0,144,18.. or in case of lzw probably something like 0,256,257,258, but no such luck.

The images may be encrypted somehow, although I wonder why they'd bother, as nothing else is encrypted and even the image header has the 320 and 200 values are unencrypted..

One thing to try is to try changing one of the values randomly and see how it affects the image; might give some clues as to whether it's checksumming or doing some form of RLE.

If only we could ask Sid and the team directly (and if only they could remember!)

Good idea. At least it's less work than disassembling the whole thing..

Who knows =).

I'm using GENDER.PIC because it's one of the smallest (if not the smallest) and it has a nice black edge - first 19 pixels are black.

What I found was that changing the data in offsets 0x06-0x16 has no effect (which also means that my variable-width-dumps are worthless). The byte at offset 0x17 did, however, do something. If I change the byte from 0x55 to 0x56, we get:

http://dl.dropboxusercontent.com/u/75238938/ca_pic1.png

The result is interesting in two ways:
One, the pixel format is 4-bit, with one "pixel byte" representing two pixels. Second, since the pattern gets repeated throughout the picture (even though I only changed two bits!) says that they're using some LZ algorithm, again, probably LZW.

Time to do more word-width dumps, I guess =)

edit: The 16 bytes that had no effect? they're mapping information for cga and possibly others. If I zero them out, cga images are black, while vga, mcga and ega images are unchanged. (tandy crashes dosbox =)

edit: 0x0b at offset 0x16 is the max bit width of lzw codepoint (0xb = 11 bits) If I change this to 0x0a, for instance, the images get garbled at some point (when the lzw engine resets itself ahead of time).

The first byte of the PIC files I thought were a version? Apparently it's some kind of info flags field. For 0x0F, the lzw data starts (with the 0x0b byte) at offset 0x16, while for 0x07, the 0x0b byte is in offset 0x06, right after the width and height fields.

Ok, the Crime files!

These files are split into four parts: Header, Participants, Events, and Objects.

Header (4 bytes):

Byte 0 - Holds the number of participants
Byte 2 - Holds the number of Events + 1 -- probably used to determine the offset for the objects

Bytes 1 & 3 are 00 (probably filler)

Participants (48 bytes per person):
Now this has a lot of fields I'm not sure about. From playing about with them, they seem to denote things like which city they're in; which organization; and possibly where they were recruited? Needs more testing here!

Byte 0-1 -- Delimiter set to FF
Byte 2 - does something...
Byte 3 - 00 filler
Byte 4-35 - This is the name of the participants role
Byte 36-37 - again, does something...
Byte 38 - Mastermind flag. Set to 00 for a normal person, 01 for mastermind. If no-one is set to 00, then the game will works ok! Haven't tried with two masterminds however
Byte 39 - another strange thing...
Byte 40-41 - 00 filler
Byte 42 - yet another variable...
Byte 43 - 00 filler
Byte 44 - Rank of Participant, used solely for scoring
Byte 45 - 00 filler
Byte 46 - a final strange variable
Byte 47 - 00 filler

The thing with the variables is that it all works fine if I zero them. So not quite sure what they do yet. If I get more info, then I'll fill it in!

Events (44 bytes per event):

These are the events that make the crime tick. The whole crime is basically set from this; essentially it's the brains of the game. One thing to note is that this is relatively simple, and the game itself determines a lot of the links. Like this data doesn't say what order each event happens; instead the game determines the ordering based on some internal logic. I'll post later my current understanding of this logic. You can create events which break the logic and the game will still work. However, it skip the broken events, which may then break other events due to them not being able to happen.

Each event is one of 7 actions, which each has some text description, a person who is doing the event, possibly a person who is having the event happen to, and then some flags.

Ok, let's get down to the action.

Byte 00 - Person doing the event (so person sending a message or committing a crime)
Byte 01-03 - 00 filler
Byte 04 - Message ID. This is used to link two related events together (eg, sent a message and received message events) and is also used to determine what text from clues.txt to display for this event. If unrelated events share an ID, then one of the events will not happen! It also appears that you can set a message to be sent from two different people using the same ID, and the game will randomly pick between them.
Byte 05 - 00 filler
Byte 06-36 - Description of event, padded with 00
Byte 37 - 00 filler
Byte 38 - Person the event is happening to (00 for crimes)
Byte 39 - Type of event. Main ones are: 02-Sent message, 03-Received message, 08-Met with, 09-Was met by, 20-Crime. There needs to be a pair for 02/03 and 08/09 (so can't send a message without having a receive message set for the same ID and the same people, but swapped). There are also two events for sending and receiving a package which is only used for one crime. I'll put the ids for these in later
Byte 40 - Bit-mask of items received by the person doing the event. Ie, 1 for item 1, 2 for item 2, 4 for item 3, and 8 for item 4. You can gain multiple items by an event this way! If the item already exists in someone elses possession, then it'll move instantly to the person doing this event.
Byte 41 - Bit-mask of items destroyed by the event. Used usually in crimes
Byte 42 - Score of event. This is a value from 00 to 0A, and is multipled by 40 to give the score you lose by having the event happen (or gain by stopping it). For a crime to count as succeeded, events have to happen with a combined score of 0A. This was the cause of the ultimate plot bug.
Byte 43-44 - 00 filler

Note, Internally the game doesn't know the difference between a minor crime (like withdrawing money) and a major one. Also there's no timing information in the events. I'll give my thoughts on the logic later.

Objects (18 bytes each) -- note there is 44 bytes of just padding - starting with FF then 43 bytes of 00:

Byte 0-16 - Name of object
Byte 17 - Picture ID of object
Byte 18 - FF delimiter

And that's it. The picture IDs are:

Processed Drugs 00
Sniper Rifle 01
Weapons 01
Raw Drugs 02
Chemicals 03
Supplies 03
Explosives 04
Alarm Bypass 05
Product 05
Photographs 06
Hostage 07
Pilot 07
Escapee 07
Bomb 08
Payoff 09
$1,000,000 09
Gold Coin 09
$1,000,000 09
Photographs 0A
Camera 0B
Blueprints 0C
Mission Plans 0C
Stolen Goods 0D
Jeep 0D
Sample Item 0D
Forgery 0D
Equipment 0D
Passwords 0E

-----

And that's the crime file! Logic for events to come!

Crime Logic

The game seems to work on the overal schedule of:

1) The 'organizer' (usually first person on the list of participants) sends messages to each other participant, usually letting them know they have to do some evil

2) Minor crimes are committed - usually to get items

3) These items are given to other people, either through meetings or messages

4) The people with the items then commit the 'major crime' (just another crime, but one which uses up items and gives score)

5) The people who have done the major crimes then tell someone they've succeeded.

And the logic seems to require this rough order. So a person won't be able to commit a 'crime' (including withdrawing money) until they have been messaged first. In fact, if they aren't messaged, the game will ignore the crime events totally.

So, the game seems to do the messages which don't require or generate items first, in the rough order of first participants messages, then second participants, and so on. After this, crimes which generate items (but don't require items) are committed. Then the game will run all messages which deal with that item, as long as the person who is sending the message has the item. It will deal with messages chains, where items are passed from person A->person B->person C in that order. It also deals with meetings in a similar fashion (I'm not sure if meetings occur after messages, or if they are mixed). When someone is set to do a crime that requires/uses up an item, this will only occur after they get it (other people might still be doing the item transfer chain in the meantime). Then it goes back to messages/meetings that deal with any new generated items.

Once all events that can occur have, the mission is over.

That's the general gist - but there seems to be some other nuances involved. It looks like you can't have two identical types of events happening between the same people (so no A-messages->B, then later A-messages->B - unless items are invoved in one of them to make it unique).

Any more findings, and I'll put them in here
BTW, Intro.exe is the graphics/sound/joystick selection pages at the start