Cave Story+ Linux/Windows - incompatible saves?

Discussion in 'Technical Issues' started by Theon144, Dec 25, 2011.

Old topic!
The last post in this thread is over 60 days old. Posting in this thread will be considered a bump, so please make an attempt to be courteous if you go ahead with it.

If the last post is over 6 months old, it may instead be a better idea to start a new topic. If you aren't sure about what to do, feel free to ask a staff member for help, or try to locate a 'general questions'-type thread if it exists in this (sub-)forum.
  1. Dec 25, 2011
    Theon144
    Neophyte Member
    "Fresh from the Bakery"
    Join Date: Dec 18, 2011
    Location:
    Posts: 2
    I use Linux at home, but I happened to be on a Windows machine when I started playing Cave Story+, I played for some time, then I backed up the whole folder on my thumb drive, and copied the profile.dat file into the Cave Story folder on my Linux computer, but then it says the save file is corrupt - so I also copied the rest of the folder (i.e. the data folder), but it still says the save file is corrupt. How can I fix this? Are the versions incompatible?
     
  2. Dec 25, 2011
    Noxid
    Pretty Skater in a Sailor Suit
    "Life begins and ends with Nu."
    Join Date: Aug 28, 2009
    Location: space or some shit
    Posts: 5188
    It depends, is linux little-endian like mac? If so then maybe.
     
  3. Dec 26, 2011
    andwhyisit
    Administrator
    "Life begins and ends with Nu."
    Join Date: Jul 15, 2007
    Location: Australia
    Posts: 5635
    Age: 29
    Can you give me a linux save to mess around with? The later on in the game you have it the better. Also please make sure that you save in a map that has background music and that you have at least two slots saved to (no conditions for the second save).
     
  4. Dec 27, 2011
    Theon144
    Neophyte Member
    "Fresh from the Bakery"
    Join Date: Dec 18, 2011
    Location:
    Posts: 2
    I'm not sure, I think it depends on the kernel/architecture. Anyway, that should be easily fixed by reversing the bit order, right?

    Sorry, the save that I spent few hours on is the windows one, so the Linux one is almost at the very beginning - though I guess I could give it to you, why does it need to be later in the game?
     
  5. Dec 27, 2011
    Ambient_Malice
    Senior Member
    "Huzzah!"
    Join Date: Dec 7, 2011
    Location: Australia. In Bob Katter's secret techno-dungeon.
    Posts: 206
    Age: 24
    Most likely, he wants more flags set, for reference.
     
  6. Dec 27, 2011
    andwhyisit
    Administrator
    "Life begins and ends with Nu."
    Join Date: Jul 15, 2007
    Location: Australia
    Posts: 5635
    Age: 29
    @Ambient_Malice: Items, weapons, teleporter information, equip information, xp, whimsical star info, etc. None of these appear in a new save. Flags on the other hand are saved the same on all systems regardless of byte ordering.

    @Theon144: If you can get as far as activating the teleporter in Arthur's house and saving in a room that has music in it then I can work with it. No guarantees that I'll be able to get to it before I go away to the beach for a few days but I'll see what I can do. Also, please send me the windows save you want to convert as well.
     
  7. Dec 27, 2011
    Ambient_Malice
    Senior Member
    "Huzzah!"
    Join Date: Dec 7, 2011
    Location: Australia. In Bob Katter's secret techno-dungeon.
    Posts: 206
    Age: 24
    My mistake. My definition of 'flag' was a bit off. Thanks for setting me straight.
     
  8. Dec 31, 2011
    Maxim
    Neophyte Member
    "Fresh from the Bakery"
    Join Date: Dec 31, 2011
    Location:
    Posts: 9
    I'm interested in this as well. More curiosity than anything else. I attempted to do the same thing when I first started playing it.

    When the game loads with the Windows game save, it concludes that the game save is "corrupt" and can only be deleted, or continue the game without being able to save.

    In the console it returns this error message:
    Save File: Wrong size! (131104 != 130452)

    It's obvious that the files are different lengths but to my layman eyes they appear to begin the same. I attempted to truncate the Windows game save with dd, but the game didn't save any of the configs (controls), or any of the game saves (all of the slots were "New"). So it appears to me something out of my scope and not just a file size issue.

    I'll try and get a game going with the requested depth into the game this weekend.

    Ok. I've played the game to the point where it was requested of the other user above.

    I've uploaded the game saves here.

    I've included Windows and Linux versions. Each of the save slots is used, with each to the same point, with the same life, and items, and each in the same room. They should be all very very close to the same.
     
  9. Jan 18, 2012
    Maxim
    Neophyte Member
    "Fresh from the Bakery"
    Join Date: Dec 31, 2011
    Location:
    Posts: 9
    Is there something that I can do having no significant experience with hex editing?

    I found a converter script which probably does what needs to be done but is only applicable for Mac save files and doesn't appear to work properly with Cave Story + saves.

    http://www.cavestory.org/forums/threads/1361/

    Also I think it requires a plist file to exist before editing the save, so I don't have that in order to be able to use it.

    If it does work properly I could probably cut out the save file section using dd, and save it as a file. I'd just have to figure a way to implant it back in to convert it back to a Windows save. At that point I could go back and forth pretty easily.
     
  10. Jan 27, 2012
    andwhyisit
    Administrator
    "Life begins and ends with Nu."
    Join Date: Jul 15, 2007
    Location: Australia
    Posts: 5635
    Age: 29
    Alright, I've figured it out, pass me the file you want converted. Sorry for the delay btw.
     
  11. Feb 6, 2012
    Maxim
    Neophyte Member
    "Fresh from the Bakery"
    Join Date: Dec 31, 2011
    Location:
    Posts: 9
    I've posted a Windows game save here.


    It's much further along than the others.

    Is this a process that I can execute myself to convert from Linux to Windows and back again?
     
  12. Feb 12, 2012
    andwhyisit
    Administrator
    "Life begins and ends with Nu."
    Join Date: Jul 15, 2007
    Location: Australia
    Posts: 5635
    Age: 29
    Let me know if this works:
    http://www.mediafire.com/?9d5md4ujrmhkez4
     
  13. Feb 12, 2012
    Maxim
    Neophyte Member
    "Fresh from the Bakery"
    Join Date: Dec 31, 2011
    Location:
    Posts: 9
    This appears to work perfectly. It loaded in both CaveStory+, and CaveStory+_64. I'm not sure of the version numbers.

    I loaded it up and ran around a little. What's the process involved in making this conversion?
     
  14. Feb 14, 2012
    andwhyisit
    Administrator
    "Life begins and ends with Nu."
    Join Date: Jul 15, 2007
    Location: Australia
    Posts: 5635
    Age: 29
    One's a 32-bit executable, and the other a 64-bit executable.
     
  15. Feb 15, 2012
    Maxim
    Neophyte Member
    "Fresh from the Bakery"
    Join Date: Dec 31, 2011
    Location:
    Posts: 9
    Is making this conversion something I can do myself?
     
  16. Feb 16, 2012
    andwhyisit
    Administrator
    "Life begins and ends with Nu."
    Join Date: Jul 15, 2007
    Location: Australia
    Posts: 5635
    Age: 29
    I'm not sure how to explain it, so no unfortunately. Just let me know when you are ready and I'll convert it back.
     
  17. Feb 16, 2012
    duncathan
    .
    "Life begins and ends with Nu."
    Join Date: Oct 18, 2011
    Location:
    Posts: 2280
    Age: 15
  18. Feb 18, 2012
    Maxim
    Neophyte Member
    "Fresh from the Bakery"
    Join Date: Dec 31, 2011
    Location:
    Posts: 9
    Using the code from the conversion tool, written by Celtic Minstrel, and an application called VBinDiff I've been able to create a save game exactly identical to the one you posted.

    My concern is that since I have to drop 652 bytes from the Windows game save (131104 != 130452), and I don't know what bytes are important.

    Here's the code that I use:
    Code:
    #!/usr/bin/python
    import sys
    import os
    orig = file('Profile.Windows.2012-01-01.dat','rb')
    new = file('Profile.new.dat','wb')
    def copy(n):
    new.write(orig.read(n))
    def swap(n):
    temp = orig.read(n)
    temp2 = []
    for c in temp: temp2 += c
    temp2.reverse()
    temp = ''.join(temp2)
    new.write(temp)
    #0000
    copy(8)   # 000-007 | header or something
    copy(4)   # 008-00B | current map
    copy(4)   # 00C-00F | current song?
    #0010
    copy(4)   # 010-013 | horizontal pos
    copy(4)   # 014-017 | vertical pos
    copy(4)   # 018-01B | facing direction
    copy(2)   # 01C-01D | max HP
    copy(2)   # 01E-01F | whimsical star
    #0020
    copy(2)   # 020-021 | Current HP
    copy(2)   # 022-023 | Unused? Possibly alignment filler
    copy(4)   # 024-027 | Current Weapon
    copy(4)   # 028-02B | Unused?
    copy(4)   # 02C-02F | Equipped Items
    #0030
    copy(4)   # 030-033 | Unused?
    copy(4)   # 034-037 | Time
    copy(4)   # 038-03B | Slot 1 Weapon Type
    copy(4)   # 03C-03F | Slot 1 Weapon Level
    #0040
    copy(4)   # 040-043 | Slot 1 Weapon Energy
    copy(4)   # 044-047 | Slot 1 Weapon Max Ammo
    copy(4)   # 048-04B | Slot 1 Weapon Current Ammo
    copy(4)   # 04C-04F | Slot 2 Weapon Type
    #0050
    copy(4)   # 050-053 | Slot 2 Weapon Level
    copy(4)   # 054-057 | Slot 2 Weapon Energy
    copy(4)   # 058-05B | Slot 2 Weapon Max Ammo
    copy(4)   # 05C-05F | Slot 2 Weapon Current Ammo
    #0060
    copy(4)   # 060-063 | Slot 3 Weapon Type
    copy(4)   # 064-067 | Slot 3 Weapon Level
    copy(4)   # 068-06B | Slot 3 Weapon Energy
    copy(4)   # 06C-06F | Slot 3 Weapon Max Ammo
    #0070
    copy(4)   # 070-073 | Slot 3 Weapon Current Ammo
    copy(4)   # 074-077 | Slot 4 Weapon Type
    copy(4)   # 078-07B | Slot 4 Weapon Level
    copy(4)   # 07C-07F | Slot 4 Weapon Energy
    #0080
    copy(4)   # 080-083 | Slot 4 Weapon Max Ammo
    copy(4)   # 084-087 | Slot 4 Weapon Current Ammo
    copy(4)   # 088-08B | Slot 5 Weapon Type
    copy(4)   # 08C-08F | Slot 5 Weapon Level
    #0090
    copy(4)   # 090-093 | Slot 5 Weapon Energy
    copy(4)   # 094-097 | Slot 5 Weapon Max Ammo
    copy(4)   # 098-09B | Slot 5 Weapon Current Ammo
    copy(4)   # 09C-09F | Slot 6 Weapon Type
    #00A0
    copy(4)   # 0A0-0A3 | Slot 6 Weapon Level
    copy(4)   # 0A4-0A7 | Slot 6 Weapon Energy
    copy(4)   # 0A8-0AB | Slot 6 Weapon Max Ammo
    copy(4)   # 0AC-0AF | Slot 6 Weapon Current Ammo
    #00B0
    copy(4)   # 0B0-0B3 | Slot 7 Weapon Type
    copy(4)   # 0B4-0B7 | Slot 7 Weapon Level
    copy(4)   # 0B8-0BB | Slot 7 Weapon Energy
    copy(4)   # 0BC-0BF | Slot 7 Weapon Max Ammo
    #00C0
    copy(4)   # 0C0-0C3 | Slot 7 Weapon Current Ammo
    copy(4)   # 0C4-0C7 | Slot 8 Weapon Type
    copy(4)   # 0C8-0CB | Slot 8 Weapon Level
    copy(4)   # 0CC-0CF | Slot 8 Weapon Energy
    #00D0
    copy(4)   # 0D0-0D3 | Slot 8 Weapon Max Ammo
    copy(4)   # 0D4-0D7 | Slot 8 Weapon Current Ammo
    copy(4)   # 0D8-0DB | Inventory R1C1
    copy(4)   # 0DC-0DF | Inventory R1C2
    #00E0
    copy(4)   # 0E0-0E3 | Inventory R1C3
    copy(4)   # 0E4-0E7 | Inventory R1C4
    copy(4)   # 0E8-0EB | Inventory R1C5
    copy(4)   # 0EC-0EF | Inventory R1C6
    #00F0
    copy(4)   # 0F0-0F3 | Inventory R2C1
    copy(4)   # 0F4-0F7 | Inventory R2C2
    copy(4)   # 0F8-0FB | Inventory R2C3
    copy(4)   # 0FC-0FF | Inventory R2C4
    #0100
    copy(4)   # 100-103 | Inventory R2C5
    copy(4)   # 104-107 | Inventory R2C6
    copy(4)   # 108-10B | Inventory R3C1
    copy(4)   # 10C-10F | Inventory R3C2
    #0110
    copy(4)   # 110-113 | Inventory R3C3
    copy(4)   # 114-117 | Inventory R3C4
    copy(4)   # 118-11B | Inventory R3C5
    copy(4)   # 11C-11F | Inventory R3C6
    #0120
    copy(4)   # 120-123 | Inventory R4C1
    copy(4)   # 124-127 | Inventory R4C2
    copy(4)   # 128-12B | Inventory R4C3
    copy(4)   # 12C-12F | Inventory R4C4
    #0130
    copy(4)   # 130-133 | Inventory R4C5
    copy(4)   # 134-137 | Inventory R4C6
    copy(4)   # 138-13B | Inventory R5C1
    copy(4)   # 13C-13F | Inventory R5C2
    #0140
    copy(4)   # 140-143 | Inventory R5C3
    copy(4)   # 144-147 | Inventory R5C4
    copy(4)   # 148-14B | Inventory R5C5
    copy(4)   # 14C-14F | Inventory R5C6
    #0150
    copy(4)   # 150-153 | Inventory R6C1
    copy(4)   # 154-157 | Inventory R6C2
    copy(4)   # 158-15B | Slot 1 Teleporter Menu
    copy(4)   # 15C-15F | Slot 1 Teleporter Location
    #0160
    copy(4)   # 160-163 | Slot 2 Teleporter Menu
    copy(4)   # 164-167 | Slot 2 Teleporter Location
    copy(4)   # 168-16B | Slot 3 Teleporter Menu
    copy(4)   # 16C-16F | Slot 3 Teleporter Location
    #0170
    copy(4)   # 170-173 | Slot 4 Teleporter Menu
    copy(4)   # 174-177 | Slot 4 Teleporter Location
    copy(4)   # 178-17B | Slot 5 Teleporter Menu
    copy(4)   # 17C-17F | Slot 5 Teleporter Location
    #0180
    copy(4)   # 180-183 | Slot 6 Teleporter Menu
    copy(4)   # 184-187 | Slot 6 Teleporter Location
    copy(4)   # 188-18B | Slot 7 Teleporter Menu
    copy(4)   # 18C-18F | Slot 7 Teleporter Location
    #0190
    copy(4)   # 190-193 | Slot 8 Teleporter Menu
    #0194
    copy(4)   # 194-197 | Slot 8 Teleporter Location
    #0195
    copy(1132) # 0196-0600 / Copy these...
    orig.read(4) # Dropping 4 bytes from the end here.
    #0604
    copy(59288)  #0604-ED9B / Copy these here
    orig.read(644) # Drop 644 bytes ????
    #1ED98-1FD93
    copy(4091) #ED9C-FD96
    copy(65533) #FD97-1FD90
    orig.read(4) #Drop last four bytes
    # And the remainder of the file:
    #new.write(orig.read())
    orig.close()
    new.close()
    print "--> Profile converted."
    I found that none of the bytes need swapped, however, there are sections which are longer. For those I simply just dropped the bytes from the end of whatever section in the Windows file. I don't know, however, if anything is stored in those bytes.

    The section where everything is listed out in the beginning could be summed up with a single copy command. It could be broken down into 3 sections, with dropping bytes at the end of each. I also need to figure out how to switch it back to a Windows game save, but I think that'll be for another day. It should be fairly easy if I can just fill the space with zeros where I dropped the bytes originally.
     
  19. Feb 19, 2012
    andwhyisit
    Administrator
    "Life begins and ends with Nu."
    Join Date: Jul 15, 2007
    Location: Australia
    Posts: 5635
    Age: 29
    This is the heart of the matter and why I didn't post any instructions or conversion tools in the first place.

    I'll see if I can ask tyrone for a rom map of the CS+ Profile.dat formats. Or at least a list of bytes to remove.
     
  20. Feb 19, 2012
    simon
    Neophyte Member
    "Fresh from the Bakery"
    Join Date: Feb 19, 2012
    Location:
    Posts: 2
    Unfortunately the save game handling in Cave Story is a bit messed up. Instead of proper serialization, it just uses something like
    Code:
    len = sizeof(SaveData);
    fread(&gSaveData, 1, len, file);
    
    SaveData is a C struct consisting of lots of arrays and other structs.
    Everyone familiar with C structures knows that alignment is treated differently depending on the compiler used. In particular, GCC seems to use a maximum alignment of 4 bytes while MSVC uses 8 bytes.

    Profile.dat starts with an array of 81 saved game states (27 mods, 3 slots each). In the Windows version, those are 1568 bytes each, while in the Linux version, the size is 1560. Finally, the total struct size is padded by an additional 4 bytes (the last 4 bytes in the file) in Windows.
    Summing up, 8*81 + 4 = 652, which explains the difference.

    I hope this explanation makes at least some sense. The following code should work, although I didn't have the chance to test it. I'm running Linux only at the moment. If someone could post Profile.dat from Windows, that would be very appreciated. I'm particularly interested in a Profile.dat where several of the 81 slots are filled with actual savegames, with different difficulty (easy/normal/hard) settings. A file where only the first slot has been used is not really suitable for testing.

    Code:
    #!/usr/bin/python
    import sys
    import os
    orig = file('Profile.Windows.2012-01-01.dat','rb')
    new = file('Profile.new.dat','wb')
    def copy(n):
    new.write(orig.read(n))
    for i in range(81):
    copy(1540)
    orig.read(4)
    copy(20)
    orig.read(4)
    copy(160)
    copy(3932)
    orig.read(4)
    orig.close()
    new.close()
    HTH