PiyoPiyo File Format

Discussion in 'Music' started by gamemanj, Apr 12, 2015.

Old topic!
The last post in this thread is over 60 days old. Posting in this thread will be considered a bump, so please make an attempt to be courteous if you go ahead with it.

If the last post is over 6 months old, it may instead be a better idea to start a new topic. If you aren't sure about what to do, feel free to ask a staff member for help, or try to locate a 'general questions'-type thread if it exists in this (sub-)forum.
  1. Apr 12, 2015
    gamemanj
    Senior Member
    "This is the greatest handgun ever made! You have to ask yourself, do I feel lucky?"
    Join Date: Aug 2, 2014
    Location: In a world full of people who want every piece of
    Posts: 111
    (Note: This probably won't be updated further, if you need a decent reference see PiyoPiyoJ's gabien.music package)
    I've worked out most of the PiyoPiyo file format.
    Enough that a player can be written, at least.
    So I figured I should post it here, since it's useful for those that want to use PiyoPiyo outside Ikachan.
    Note that "int32-le" is a 32-bit Little Endian integer.
    First 4 bytes are "PMD" followed by 0x80(q3hardcore says that the 0x80 byte is a "writable" flag).
    The next 4 bytes are an int32-le for position in file of Track 1 data.
    The next 4 bytes are an int32-le for the "music wait"(milliseconds per frame)
    The next 4 bytes are an int32-le for the "loop start".
    The next 4 bytes are an int32-le for the "loop end".
    The next 4 bytes are an int32-le for the amount of 4-byte records per track.
    Track Header Data(20 bytes followed by 256 bytes, then 64 bytes):
    The first byte is the "Octave" field
    The second byte is the "Icon" field.
    The next 2 bytes are unknown.
    The next 4 bytes are an int32-le for the "Length" field(time envelope is spread over, assume to be 11025==1 second)
    The next 4 bytes are an int32-le for the "Volume" field.
    The next 8 bytes are unknown.
    The next 0x100 bytes are the waveform(8-bit, signed. The sample rate is handled, for me at least, by changing the sample rate entered into the calculation below.)
    The next 0x40 bytes are the envelope(also 8-bit, also apparently unsigned)
    (This repeats 3 times, track P doesn't have this data)
    Finally, there's an int32-le for Track P's volume.
    That's the end of the header.
    Then, at the offset mentioned, the track data begins.
    The track data is not interleaved-it's Track 1 followed by Track 2 followed by Track 3 followed by Track P.
    All the tracks are made of 4-byte records.
    The 4-byte records are:
    First 3 bytes are a bitfield(lowest bit/first byte is lowest pitch),
    last byte is the Pan.(0 for no change, else 1 to 7)
    The amount of these records per track is in the header.
    The calculation you need to turn the note position and octave into the relative pitch is:
    First, pos must be 0-based, with higher pitches being higher values.
    Then add the octave * 12 to pos.
    Run it through this:
    double relativePitch=8363d*Math.pow(2.0d,pos/12.0d)/sample_rate;
    And you have the relative pitch(where 1.0d is to play at normal speed,2.0d is to play at double speed,0.5d is to play at half speed, you get the idea)!
    (Thanks to liborganya, which is where I got that calculation from)
    As for the volume, it's weird, and I only have guesswork for now.
    If you want more detail, just look in the last thing I uploaded with a PiyoPiyo decoder in it.
    You'll probably see these put to use later today done.
    --gamemanj
     
  2. Apr 12, 2015
    Bionicobot
    Surprisingly not single anymore.
    Bobomb says: "I need a hug!"
    Join Date: Aug 16, 2014
    Location: Earth
    Posts: 850
    Age: 16
    I clap for you, sir. This is amazing.
     
  3. Apr 12, 2015
    gamemanj
    Senior Member
    "This is the greatest handgun ever made! You have to ask yourself, do I feel lucky?"
    Join Date: Aug 2, 2014
    Location: In a world full of people who want every piece of
    Posts: 111
    Okay, the latest IkachanMapEdit, V15 (EDIT2: V15.1, which lowers CPU usage and fixes Pan), has a (hopefully, I've tested it) fully working PiyoPiyo player in it.
    So if you need source for a Java PiyoPiyo player, you should find it in (EDIT2: The gabien.music package, but you'll need the rest of gabien for it to play to)
    For future reference(EDIT:for when people start looking at other parts of Ikachan Map Editor for whatever reason): I *know* that the code around MainGame and ChatMinorState is messy.
    (EDIT2: I just fixed Pan, use V15.1)
     
  4. Apr 13, 2015
    q3hardcore
    Senior Member
    "Ha! Ha! Ha! Mega Man is no match for my Mimiga Man!"
    Join Date: Jan 22, 2015
    Location:
    Posts: 231
    Thanks for this. If any of the unknown values are always 00, they might just be padding.

    EDIT: the byte that is normally 0x80, I think if it's any other value, the file is treated as being read-only.
     
  5. Apr 16, 2015
    q3hardcore
    Senior Member
    "Ha! Ha! Ha! Mega Man is no match for my Mimiga Man!"
    Join Date: Jan 22, 2015
    Location:
    Posts: 231
  6. May 1, 2015
    gamemanj
    Senior Member
    "This is the greatest handgun ever made! You have to ask yourself, do I feel lucky?"
    Join Date: Aug 2, 2014
    Location: In a world full of people who want every piece of
    Posts: 111
    And now, an editor!
    This is version 2.1.
    View attachment PiyoPiyoJ2.1.zip
    Note that if piyoDrums.bin is missing, drums will be silent.
    Also, it will warn you if this occurs.
    Still not sure how good playback is. Volume *seems* to be mostly the same, pan is untested, but speed seems fine.
    Again, all experimental.
     
  7. May 6, 2015
    Jazz Jackalope
    Senior Member
    "Ha! Ha! Ha! Mega Man is no match for my Mimiga Man!"
    Join Date: Jul 29, 2009
    Location: The Surface
    Posts: 236
    Hell YES. Even using the program that forces PiyoPiyo to be windowed, I could never get the cursor to align properly. A new editor is a boon. Thank you!
    edit: Using the auto double post combiner like a bawss.

    Okay, after an initial fiddling, I have some thoughts.

    Firstly, the music playback is identical to Piyo Piyo/PPPlayer, and this is great. Playback was unfortunately a bit slower than it should be, but I'm sure that's easily fixed.

    Now, the UI... Very utilitarian. A bit too much so. You don't necessarily need the fancy icons and note shapes of the Chinese puzzle box that is PP (and the hostile spirit inside it), but some things would improve it greatly:

    -A lettered keyboard in view at all times. This is too invaluable for any music program.
    -Use of the arrow keys for going right/left in the song. Or that thing with the mouse at the edges moving from measure to measure. Setting the playback postion. Something. Anything.

    Jumping to the end or start of the song is a great feature. And this is still loads preferable to PP just for the fact I can freely bring up sheet music to look at.
     
  8. May 19, 2015
    gamemanj
    Senior Member
    "This is the greatest handgun ever made! You have to ask yourself, do I feel lucky?"
    Join Date: Aug 2, 2014
    Location: In a world full of people who want every piece of
    Posts: 111
    The speed of the music playback...I *still* haven't worked out what's going on with that. If anyone could find the exact way that works, do tell me. At first I assumed the delay was in milliseconds, but now it seems it's Pixel Mystery Timing Units. I'll probably use CE to find the timing code, so I can work out what on earth is going on.(EDIT1: As it turns out, the delay is SUPPOSED to be milliseconds, but when I tested official PiyoPiyo against PiyoPiyoJ by recording the output of both, PiyoPiyoJ was running too fast, so I slowed it down. The next version will use strictly 1000ms, for better or worse.)

    The UI was written with the approach of "The less stuff there is on it, the less code I need to write". I'll add the arrow keys and the lettered keyboard - I didn't know how useful they were.

    I'll probably also add the ability to control looping(even though all PiyoPiyo stuff I know of apart from Pixel's editor uses looping, it's easier to write music when the loop boundaries aren't around),
    and make the program optimize the file size upon saving(I should have done this ages ago TBH).

    Sorry about the late response, I didn't see someone had replied to this thread...

    I'll leave an EDIT here when I'm done.
    (EDIT2: Done!)
     
  9. May 19, 2015
    Jazz Jackalope
    Senior Member
    "Ha! Ha! Ha! Mega Man is no match for my Mimiga Man!"
    Join Date: Jul 29, 2009
    Location: The Surface
    Posts: 236
    S'aight, s'aight. Much better! The speed is just a bit fast now, but considering this is such a fiddly thing I think it's suitable for production purposes as is. Only a few notes end up quick, and it could honestly just be my computer chocking on it and a memory-hog browser at the same time.

    The UI additions are a great help, sir, not regrettable in the least. Especially when moving the loop points around. I think all the waist-high usage issues are fixed now. Thanks a bunch!
     
  10. Jun 12, 2015
    ShinigamiMachine
    Senior Member
    "This is the greatest handgun ever made! You have to ask yourself, do I feel lucky?"
    Join Date: Sep 19, 2013
    Location: C:\Users\Shinigami\Dirty_Videos
    Posts: 124
    Age: 19
    This is really cool!

    Also, Jazz, how did you make PiyoPiyo windowed?
     
  11. Jun 12, 2015
    q3hardcore
    Senior Member
    "Ha! Ha! Ha! Mega Man is no match for my Mimiga Man!"
    Join Date: Jan 22, 2015
    Location:
    Posts: 231
    You could use anything that supports running old DirectX apps as windowed..
    This would likely work: http://sourceforge.net/projects/dxwnd/

    But PiyoPiyo really doesn't work properly when running in a window.
     
  12. Jun 13, 2015
    ShinigamiMachine
    Senior Member
    "This is the greatest handgun ever made! You have to ask yourself, do I feel lucky?"
    Join Date: Sep 19, 2013
    Location: C:\Users\Shinigami\Dirty_Videos
    Posts: 124
    Age: 19
    Oh alright, thanks anyway! PiyoPiyo runs really choppy and at varying speeds in fullscreen on my computer.
     
  13. Jul 7, 2015
    Jazz Jackalope
    Senior Member
    "Ha! Ha! Ha! Mega Man is no match for my Mimiga Man!"
    Join Date: Jul 29, 2009
    Location: The Surface
    Posts: 236
    Well... I've kind of got an awkward thing to share.

    The latest version of DXWnd can run PiyoPiyo *perfectly*. No funky colors, cursor goes in the right position... Though I didn't get it to let other windows be on top of it yet.

    But! I don't think this project was a waste of time. Sure, PP is far more delightful to look at, but some of its functions just get obscured in its cleverness. I mean, look at this:

    [​IMG]

    What the hell do those things do? As far as I can tell, the Mac-like windows prevent editing, and the burst... puts you back on the note placement tool. I don't understand.

    Plus there's the annoying fact that you can only work on one song for the entire session, which blows all the minor demons in the seventh circle of hell. That alone is worth the separate program. PP also doesn't seem to let you click on the piano keys to test the note, unlike OrgMaker.

    So, the warts can be smoothed out. Speaking of, playback is still a little funky... But I really have no idea how to fix that. To make up for that, I made myself useful and added letters to the note keys. Here's an updated jar with the changed source file. Additions are added under the same license as the source.

    I might see if I can help make more UI improvements later, as time allows.
     
  14. Jul 8, 2015
    andwhyisit
    Administrator
    "Life begins and ends with Nu."
    Join Date: Jul 15, 2007
    Location: Australia
    Posts: 6057
    Age: 32
    The funky colours go away if you change your computer's display settings from 32-bit to 24-bit. I've had that written on the site for a number of years now.
     
  15. Jul 8, 2015
    Bavi_H
    Senior Member
    "This is the greatest handgun ever made! You have to ask yourself, do I feel lucky?"
    Join Date: Jul 11, 2009
    Location: Texas, USA
    Posts: 90
    These buttons are Duplicate and Clear.

    To Duplicate:
    1. Click the Pencil button, then drag in the selection bar to select a range.
    2. Click the Duplicate button, then click at a point in the piano roll to paste a copy of the selected range.

    To Clear:
    1. Click the Pencil button, then drag in the selection bar to select a range.
    2. Click the Clear button to clear the selected range.

    Here are some brief guides to PiyoPiyo:
    Go to Ikachan Modding Guide, open the spolier section, and read section 2 "Editing Music".
    Re: PiyoPiyo Tutorial? -- Some other tips about things that might be unclear.
     
    Last edited by a moderator: Sep 20, 2015
  16. Jul 8, 2015
    Jazz Jackalope
    Senior Member
    "Ha! Ha! Ha! Mega Man is no match for my Mimiga Man!"
    Join Date: Jul 29, 2009
    Location: The Surface
    Posts: 236
    Oh, I... Oh. Sorry. I'm not entirely sure if the problems with older DXwnd versions was wrong settings, I just know it suddenly works well after being updated.

    Riiiiight.... That thing which I already kept. Ooops. Still, though, the UI remains a tad unnecessarily cryptic.

    So, my object is weakened but I thank you two. For the J version it's still pretty great being able to switch songs within a session. (Of course I had to say "section", didn't I?)
     
    Last edited by a moderator: Sep 20, 2015
  17. Jul 14, 2015
    gamemanj
    Senior Member
    "This is the greatest handgun ever made! You have to ask yourself, do I feel lucky?"
    Join Date: Aug 2, 2014
    Location: In a world full of people who want every piece of
    Posts: 111
    I'll integrate this now (with "sharp" letters added, and switching how it renders the text)
    EDIT: Done.