G2 patch format deciphered

mosc · Posted: Fri Mar 17, 2006 6:01 pm Post subject:

Congrats... thumb up

blue hell · Posted: Fri Mar 17, 2006 6:35 pm Post subject:

Can confirm it to be working on patches, after trying a few. For performances it must ne a little different.

edit :

with this code :

jamos · Posted: Fri Mar 17, 2006 7:01 pm Post subject:

I suspect that the header length is different in prf2 files; I just used a fixed length in my test code, whereas the correct way to do it would be to search for the first byte after the null terminator in the header string.

blue hell · Posted: Fri Mar 17, 2006 7:07 pm Post subject:

Ok, performances work as well. Just do not start at a fixed offset of 0x50 but instead scan for the first occurance of \0, set the starting point just past that \0 then execute the crc over the rest of the bytes (but not the last two (*1)). This will be "better" for patches as well of course :)

*1 : when you do however include the last two byte in the CRC calculation the result will be zero, a known property for CRC's alowing for a somewhat easier check for correctness.

Edit : we cros posted :)

monobass · Posted: Sat Mar 18, 2006 9:38 am Post subject:

I missed this thread in the past. Good work Smile

It must make the Clavia developers chuckle reading this.. I hope it does anyway. Why they don't just give away the patch format I'll never know though... they must know it'll be reverse engineered eventually. Still.. It would be a shame to deny you guys the challenge!

blue hell · Posted: Sun Mar 19, 2006 4:57 pm Post subject:

I found a little bug in http://www.dewb.org/g2/pch2format.html regarding the Module List. As stated a module entry in the module list takes 50 bits always, but I noticed that for an OscC module 56 bits are used (so there are 6 more unknown bits) - there will undoubtedly be more exceptions. This is ugly Evil or Very Mad

A partial dump :

dasz · Posted: Sun Mar 19, 2006 7:11 pm Post subject:

and another nasty part is they may change the format at any time. But probably too easy, since it means h/w programming too. easier than dsp tho.

In Revolve days we reversed engineered the rebirt 1.5 and 2.0 file formats, and then the Propellerheads released the format for 2.0, thanks guys!

/Dasz

ian-s · Posted: Sun Mar 19, 2006 8:02 pm Post subject:

I imagine there are some changes with 1.4, exclude from mutator etc.

blue hell · Posted: Sun Mar 19, 2006 8:03 pm Post subject:

I know Dasz.

I compiled a table of the bit counts for all modules, see attachment.

Edit : attachement update.

Edit : atachment deleted, see later post on same date as last edit.

3phase · Posted: Mon Mar 20, 2006 3:42 am Post subject:

I wonder why they keep the format so secret...
Is there any harm for the product possible?

There was such nice freeware tools for the NM. I think it can be rather beneficial for the instrument when you guys succed.

mosc · Posted: Mon Mar 20, 2006 7:03 am Post subject:

I don't know what the Clavia line on this is, but I've developed programs for personal use and ones which were made public. If the level of work is 1X to develop a program, then it is 10X to support it. Some would say I'm underestimating.

During a program's life one often changes internal data structures and intermediate file formats. If these were public, then any change would have to be throughly documented and and tested - not only for whatever use the developers make of them, but for any conceivable use anyone else has devised.

I would like Clavia to continue to spend whatever resources they can devote to the G2 on improvements (new modules particularly). If they made the file format of interal operations public, then they'd have less resoruces for new stuff.

One could argue that if it were "opened up" then independent developers could make the improvements and develop new modules. I have made such arguments myself in the past. But it would then become an open source project. I love open source software, but it isn't for everyone. Such projects often diverge down separate incompatible paths; that wouldn't be very good.

Fozzie · Posted: Mon Mar 20, 2006 7:30 am Post subject:

dasz · Posted: Mon Mar 20, 2006 9:37 am Post subject:

I agree with mosc. Opening any file systems to the public means some kind of support. Like I and others keep saying, Clavia is no big shop, so any additional work is carefully considered. Each of the Clavia guys put in a LOT of time developing their products -- a lot more time than I would ever consider putting in on a regular basis.

Speaking from experience, one possble reason the G2 format is binary and not ascii (text based) is the speed of parsing the thing. A binary file loads quicker as there is minimal parsing involved. A human readable format takes longer (especially on a h/w platform). Since Clavia introduced a performance file, which is equivalent to 4 patches, this may have enforced the reason. I'm just guessing for Clavia's reasons.

well-done

I applaud this effort at reverse-engineering well-done

.
It's not easy, but it is do-able! Like I said before, we at Revolve (thanks to Frank --- aka Green Boy who did the reverse engineering) did succeed to figure out both Rebirth File format 1.5 and 2.0 with no help from Pheads.

/Dasz

Afro88 · Posted: Mon Mar 20, 2006 3:04 pm Post subject:

They don't have to open it up, document and support the patch format. They just have to leave it in the un-encrypted text format and say they don't support any use of the G2 with non-Clavia programs. Solved.

But I guess by encrypting they avoid any external programs screwing up their support.

dasz · Posted: Mon Mar 20, 2006 3:14 pm Post subject:

I'd rather have pattern memories in sequencers rather than an easier to decipher file format. And loop length changes which do not reset the sequence to 1 immediately. And muteless patch loading. And ...

/Dasz

blue hell · Posted: Mon Mar 20, 2006 7:40 pm Post subject:

A little update.

The extra length needed for some modules probably contains the the mode(s). You'll recognize a mode on that the patch needs to be recalculated when it changes. For the delay modules it's the delay time, for oscillator like things, when present, it's the wave shape. Each mode needs 6 extra data bits above the usual 50. The logic gate module is the only module having two modes, and it (indeed) needs 62 data bits. No modules have more than two modes.

In the file I posted earlier the Phaser was mentioned twice (one should be flanger), and there was a typo in a module name somewhere, I'll post an update above.

An example dump for a module having one mode.

blue hell · Posted: Tue Mar 21, 2006 4:14 pm Post subject:

Something changed in the cable list w.r. to the specs on http://www.dewb.org/g2/pch2format.html

it says :

blue hell · Posted: Wed Mar 22, 2006 7:36 pm Post subject:

Better module knowledge, no parameters yet. Present are the modes for all modules and the in/outs for about a third or so.

I'll delete the old module info file, the bit counts can be calculated from this one as 50 + mode_count * 6.

Edit : Comment( 'Learned 138 from 166, 28 to go')

Edit : Comment( 'Learned 166 from 166, 0 to go')

Edit : some error fixing, added a html version, added some params from existing module table.

The Extension 'html' was deactivated by an board admin, therefore this Attachment is not displayed.

module_table.txt

Description:

some g2 module knowledge

Download (listen)

Filename:

module_table.txt

Filesize:

42.24 KB

Downloaded:

184 Time(s)

3phase · Posted: Thu Mar 23, 2006 3:21 am Post subject:

wow..you are really working on it ...
thanks for the time you people involve in that...
I am pretty sure that once you ve suceeded there will be a practical applikation from this effort earlier or later...
good luck...

Hyde · Joined: Jul 15, 2004 Posts: 118 G2 patch files: 2

blue hell · Posted: Thu Mar 23, 2006 6:11 pm Post subject:

blue hell · Posted: Fri Mar 24, 2006 7:16 pm Post subject:

blue hell · Posted: Sat Mar 25, 2006 3:33 pm Post subject:

Did some error fixing and made a pretty printer first, that is HTML output. And I copied some parameter things from the original table.

I'll update the txt file I'll add the HTML file.

blue hell · Posted: Sat Mar 25, 2006 8:19 pm Post subject:

The bithose class has been very helpfull, youu'll want to have defined someting like

blue hell · Posted: Sun Mar 26, 2006 8:52 am Post subject:

Found another change.

In the name dump section http://www.dewb.org/g2/pch2format.html speciefies :