electro-music.com   Dedicated to experimental electro-acoustic
and electronic music
 
    Front Page  |  Articles  |  Radio
 |  Media  |  Forum  |  Wiki  |  Links  |  Store
Forum with support of Syndicator RSS
 FAQFAQ   CalendarCalendar   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   LinksLinks
 RegisterRegister   ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in  Chat RoomChat Room 
Live streaming at radio.electro-music.com

  host / artist show at your time

poster

Please visit the chat
 Forum index » Clavia Nord Modular » Nord Modular G2 Discussion
G2 patch format deciphered
Post new topic   Reply to topic Moderators: Nord Modular Editors
Page 1 of 4 [78 Posts]
View unread posts
View new posts in the last week
Mark the topic unread :: View previous topic :: View next topic
Goto page: 1, 2, 3, 4 Next
Author Message
Dewb



Joined: Apr 07, 2004
Posts: 9
Location: Boston, MA
G2 patch files: 1

PostPosted: Fri Apr 29, 2005 11:35 am    Post subject: G2 patch format deciphered Reply with quote  Mark this post and the followings unread

I'm happy to report that I've been able to reverse engineer most of the G2 .pch2 file format.

I've put a draft format spec at http://www.dewb.org/g2/pch2format.html . If the technically minded could take a look and give me clarity/readability comments I'd appreciate it.

There are some parts of the patch that I haven't figured out yet. Some I have a pretty good idea on, others I'm still in the dark. Also, each module has its own indexes for parameters, inputs, and outputs. It's not hard to document this information, but so far I've only had time to do the In/Out and Osc groups. The rest will come soon.

The original analysis was done on 1.22/1.24. I haven't checked 1.3x for changes, but I don't think it will be hard to make any necessary updates.

The inevitable follow up question is: what about a parser? Right now, I do have a super-ugly debugging parser that reads and dumps .pch2 files. I plan to write a more usable read/write parser suitable for embedding in other programs. First, though, I wanted to see what technology choices would best serve people who would like to write applications based on the parser. I'm leaning towards a straight C library but C++, Java, and Perl are also on the table.

The most obvious application is an NM1->G2 patch converter. I'd love to see one of these, however imperfect, but I don't have an NM1. I vaguely remember some discussion on the list a while back about the idea of converting both formats to and from an intermediate XML-based description language, as a way of performing the conversion. I think that's a great idea and I'm happy to work on the G2 side of that effort.

Thanks to everybody here and on the mailing list for all the patching education over the last couple years; I hope I can deliver a small bit of usefulness in return.
Back to top
View user's profile Send private message Visit poster's website
jksuperstar



Joined: Aug 20, 2004
Posts: 2503
Location: Denver
Audio files: 1
G2 patch files: 18

PostPosted: Fri Apr 29, 2005 12:06 pm    Post subject: Reply with quote  Mark this post and the followings unread

Wow! Great Work....Now we can see if some of those "hidden modules" are available!

And, we can get some of those Nord Moduler (1) tools I miss, like raw2pch.

* <- 1 gold star for you!
Back to top
View user's profile Send private message Visit poster's website
inversekinematics



Joined: Feb 12, 2005
Posts: 29
Location: Sweden
G2 patch files: 1

PostPosted: Fri Apr 29, 2005 2:00 pm    Post subject: Re: G2 patch format deciphered Reply with quote  Mark this post and the followings unread

Dewb wrote:
I'm happy to report that I've been able to reverse engineer most of the G2 .pch2 file format.

I've put a draft format spec at http://www.dewb.org/g2/pch2format.html. If the technically minded could take a look and give me clarity/readability comments I'd appreciate it.

There are some parts of the patch that I haven't figured out yet. Some I have a pretty good idea on, others I'm still in the dark. Also, each module has its own indexes for parameters, inputs, and outputs. It's not hard to document this information, but so far I've only had time to do the In/Out and Osc groups. The rest will come soon.

The original analysis was done on 1.22/1.24. I haven't checked 1.3x for changes, but I don't think it will be hard to make any necessary updates.

The inevitable follow up question is: what about a parser? Right now, I do have a super-ugly debugging parser that reads and dumps .pch2 files. I plan to write a more usable read/write parser suitable for embedding in other programs. First, though, I wanted to see what technology choices would best serve people who would like to write applications based on the parser. I'm leaning towards a straight C library but C++, Java, and Perl are also on the table.

The most obvious application is an NM1->G2 patch converter. I'd love to see one of these, however imperfect, but I don't have an NM1. I vaguely remember some discussion on the list a while back about the idea of converting both formats to and from an intermediate XML-based description language, as a way of performing the conversion. I think that's a great idea and I'm happy to work on the G2 side of that effort.

Thanks to everybody here and on the mailing list for all the patching education over the last couple years; I hope I can deliver a small bit of usefulness in return.


Impressive research there... now see if we could get some cool stuff to work in some way that it shouldn't Smile

_________________
(dx/dt=delta*(y-x))(dy/dt=r*x-y-x*z)(dz/dt=x*y-b*z)
Back to top
View user's profile Send private message Visit poster's website
King Rat



Joined: Mar 11, 2005
Posts: 98
Location: London
Audio files: 5
G2 patch files: 13

PostPosted: Fri Apr 29, 2005 6:24 pm    Post subject: Reply with quote  Mark this post and the followings unread

Eh?
Back to top
View user's profile Send private message
LLR3



Joined: Sep 10, 2004
Posts: 41
Location: Helsinki Finland

PostPosted: Sat Apr 30, 2005 7:15 am    Post subject: Reply with quote  Mark this post and the followings unread

Excellent work!

Now how about a DX7 patch converter? Anyone?

*LLR3
Back to top
View user's profile Send private message
Hyde



Joined: Jul 15, 2004
Posts: 118

G2 patch files: 2

PostPosted: Sun May 01, 2005 2:10 am    Post subject: Reply with quote  Mark this post and the followings unread

Dewb - KEEP UP THE GOOD WORK!!!

A converter would be a godsend. Using things in ways they aren't meant to be is something us glitchcore guys would love to see happen.

I have a question though: is it in any way possible for 'us' - the general public - to create our own modules? And upload them to the synth or Editor? That would be SO SICK Smile
Back to top
View user's profile Send private message
ian-s



Joined: Apr 01, 2004
Posts: 2647
Location: Auckland, New Zealand
Audio files: 42
G2 patch files: 625

PostPosted: Sun May 01, 2005 3:03 am    Post subject: Re: G2 patch format deciphered Reply with quote  Mark this post and the followings unread

excellent work!!!

Dewb wrote:
I'm leaning towards a straight C library


That would be nice.

Any thoughts on the checksum? That would seem the biggest remaining hurdle.
Back to top
View user's profile Send private message
Dewb



Joined: Apr 07, 2004
Posts: 9
Location: Boston, MA
G2 patch files: 1

PostPosted: Tue May 03, 2005 9:15 am    Post subject: Reply with quote  Mark this post and the followings unread

Thanks for the comments everyone.

All this enables is alternative ways of creating patches. Creating new modules would require writing DSP code to insert into the G2 OS package... that's a whole 'nother can of worms. Similarly, I doubt the "hidden" modules are actually available for use; the resources might be in the editor, but the code might not be in the G2.

A DX7 patch loader would be a nice test case for the parser API. Does anyone know of a site giving a good description of the DX7 .syx format?

No thoughts on the checksum yet. Still trying to fill in the unknown holes in the patch data. I've gotten a few, but one is giving me trouble.
Back to top
View user's profile Send private message Visit poster's website
jamos



Joined: Jun 01, 2004
Posts: 483
Location: Eugene, Oregon, USA
Audio files: 4
G2 patch files: 36

PostPosted: Tue May 03, 2005 9:42 am    Post subject: Reply with quote  Mark this post and the followings unread

A DX7 convertor would be a fantastic application for this. If the spec is unavailable, I have it in print form here, and can get it to anyone who needs it (will have to bum a scanner, but that should be possible).
Back to top
View user's profile Send private message Send e-mail Visit poster's website
mosc
Site Admin


Joined: Jan 31, 2003
Posts: 17825
Location: Allentown, PA
Audio files: 138
G2 patch files: 60

PostPosted: Tue May 03, 2005 12:19 pm    Post subject: Reply with quote  Mark this post and the followings unread

Dewb, you are to be commended for doing this difficult work, Congrats on the breakthrough.
_________________
--Howard
my music and other stuff
Back to top
View user's profile Send private message Send e-mail Visit poster's website AIM Address
sebber



Joined: Aug 27, 2004
Posts: 490
Location: Berlin
Audio files: 3
G2 patch files: 33

PostPosted: Wed May 04, 2005 8:15 am    Post subject: Reply with quote  Mark this post and the followings unread

thumb up I think this is just great.

If we would have a 80%-working patch-converter I'd have so much fun listening to imperfect patches which is what

a) I unintendedly do all the time
b) I like to hear most, because I like to imperfect perfect things. (they become so much more human then).

Great work. BTW, if you don't know it already, there's a pretty imperfect comparison chart for NM/G2-modules here:
http://www.sebastianberweck.de/G2/MRT.htm, maybe it's useful for a first converter.
Back to top
View user's profile Send private message Visit poster's website
nescivi



Joined: Mar 23, 2005
Posts: 94
Location: Montreal

PostPosted: Thu May 05, 2005 2:27 pm    Post subject: Reply with quote  Mark this post and the followings unread

Hi,

for what it's worth..

at the moment there is a discussion going on at the Linux Audio Developers list about a NM to PD (or something else) patch convertor.

I forwarded them the link about the G2 specs and to this discussion... as it seems you're both aiming at a similar thing.

For the archive of the LAD emaillist:
http://lalists.stanford.edu/lad/2005/04/
thread is called "Nord Modular instrument converter"
Back to top
View user's profile Send private message Visit poster's website
cappy2112



Joined: Dec 24, 2004
Posts: 2348
Location: San Jose, California
Audio files: 2
G2 patch files: 1

PostPosted: Fri May 06, 2005 11:46 am    Post subject: Reply with quote  Mark this post and the followings unread

nescivi wrote:
Hi,

for what it's worth..

at the moment there is a discussion going on at the Linux Audio Developers list about a NM to PD (or something else) patch convertor.

I forwarded them the link about the G2 specs and to this discussion... as it seems you're both aiming at a similar thing.

For the archive of the LAD emaillist:
http://lalists.stanford.edu/lad/2005/04/
thread is called "Nord Modular instrument converter"


Are you sure you got that URL correct?
Back to top
View user's profile Send private message
diskonext



Joined: Aug 26, 2004
Posts: 306
Location: London, UK

PostPosted: Fri May 06, 2005 1:18 pm    Post subject: Reply with quote  Mark this post and the followings unread

Great stuff!

Another application would be for use at the EM forum... you could make an automatic .pch2 to .jpeg convertor, for all of the non-G2 owners, and other interested folks... no need to take screenshots, just post the patch and have a picture inserted, automatically!

-diskonext

_________________
:wq
Back to top
View user's profile Send private message Visit poster's website
Blue Hell
Site Admin


Joined: Apr 03, 2004
Posts: 22226
Location: The Netherlands, Enschede
Audio files: 208
G2 patch files: 319

PostPosted: Sun May 08, 2005 12:48 pm    Post subject: Reply with quote  Mark this post and the followings unread

Wow, a nice surprise after coming home from a short holiday !

There are still some details to do of course, but this seems a good framework to start from.

BTW, I edited the link in the original post to be clickable.

Jan.
Back to top
View user's profile Send private message Visit poster's website
Blue Hell
Site Admin


Joined: Apr 03, 2004
Posts: 22226
Location: The Netherlands, Enschede
Audio files: 208
G2 patch files: 319

PostPosted: Sat May 21, 2005 4:26 pm    Post subject: Reply with quote  Mark this post and the followings unread

Right, I had completely missed the module list on my first visit to the site ... a link on top of both pages would be helpfull for dumbo's like me :-)

What tooling do you use to decipher the (module) details - seems lke you could use some help. I've been doing about the same job some years ago for the NM Classic and this took ages to complete eventhough I did get good help from others.

After the EM event I'll have some time probably to sort out a couple of modules, or some of the unknown details.

At http://www.iaf.nl/Users/BlueHell/html/nm-open/nm-open-g2info.htm I have a little bit of extra info about how patches and MIDI dumps are related to each other. This is old info from before I myself had a G2, but apart from the version numbers it should still be OK I think.

Jan.
Back to top
View user's profile Send private message Visit poster's website
BobTheDog



Joined: Feb 28, 2005
Posts: 4000
Location: England
Audio files: 32
G2 patch files: 15

PostPosted: Mon Jul 25, 2005 1:06 pm    Post subject: Reply with quote  Mark this post and the followings unread

Did this ever go any further, has the checksum been worked out yet?

Is there a similar document for the original NM format?

I would be interested on working on a converter.

Cheers

Andy
Back to top
View user's profile Send private message
Blue Hell
Site Admin


Joined: Apr 03, 2004
Posts: 22226
Location: The Netherlands, Enschede
Audio files: 208
G2 patch files: 319

PostPosted: Mon Jul 25, 2005 1:16 pm    Post subject: Reply with quote  Mark this post and the followings unread

BobTheDog wrote:

Is there a similar document for the original NM format?



http://www.iaf.nl/Users/BlueHell/info/patch303.htm

Jan.
Back to top
View user's profile Send private message Visit poster's website
BobTheDog



Joined: Feb 28, 2005
Posts: 4000
Location: England
Audio files: 32
G2 patch files: 15

PostPosted: Mon Jul 25, 2005 2:17 pm    Post subject: Reply with quote  Mark this post and the followings unread

Hi Jan,

Thanks for the info.

Andy
Back to top
View user's profile Send private message
Blue Hell
Site Admin


Joined: Apr 03, 2004
Posts: 22226
Location: The Netherlands, Enschede
Audio files: 208
G2 patch files: 319

PostPosted: Mon Jul 25, 2005 3:19 pm    Post subject: Reply with quote  Mark this post and the followings unread

You're welcome Andy, I'm sure you'll have a good time with your G2 !

Jan.
Back to top
View user's profile Send private message Visit poster's website
jamos



Joined: Jun 01, 2004
Posts: 483
Location: Eugene, Oregon, USA
Audio files: 4
G2 patch files: 36

PostPosted: Fri Mar 17, 2006 3:49 pm    Post subject: Reply with quote  Mark this post and the followings unread

Hey all;

I have a quick well-deserved break from work so I thought I'd play with the G2 patch format stuff.

The last two bytes are definitely a checksum; change them and the file won't load.

Does anyone have any ideas on the checksum algorithm? I've been monkeying with it but no luck so far.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Blue Hell
Site Admin


Joined: Apr 03, 2004
Posts: 22226
Location: The Netherlands, Enschede
Audio files: 208
G2 patch files: 319

PostPosted: Fri Mar 17, 2006 4:21 pm    Post subject: Reply with quote  Mark this post and the followings unread

jamos wrote:
I've been monkeying with it but no luck so far.


From what I've seen on checksums for the NM protocol and for the MIDI dumps for the G2 the checksums were all pretty lame, just a sum or an XOR modulo 0x80. Here however we have two bytes, that's new .... still my first gues would be a sum of some sort, not a CRC (given history).

What did you try sofar ?

I'll start making a litlle checksum checker tool right away :)

_________________
Jan
Back to top
View user's profile Send private message Visit poster's website
jamos



Joined: Jun 01, 2004
Posts: 483
Location: Eugene, Oregon, USA
Audio files: 4
G2 patch files: 36

PostPosted: Fri Mar 17, 2006 4:28 pm    Post subject: Reply with quote  Mark this post and the followings unread

Blue Hell wrote:
What did you try sofar ?


Up to now just a sum over all bytes in the entire file (less the checksum of course) and a sum over everything other than the header. Plus a CRC 16 using an algorithm I found online (and which may be faulty). I'll play with it a bit more tonight (have to work on taxes now, ugh)
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Blue Hell
Site Admin


Joined: Apr 03, 2004
Posts: 22226
Location: The Netherlands, Enschede
Audio files: 208
G2 patch files: 319

PostPosted: Fri Mar 17, 2006 5:21 pm    Post subject: Reply with quote  Mark this post and the followings unread

Do you know which polynomial the CRC used and what start value for the CRC did you try ?


Common 16 bit CRC's :

CRC-16 (IBM) (polynomial : $a001 :: x^16 + x^15 + x^2 + x^0)

CRC-CCITT-16 (polynomial : $8408 :: x^16 + x^12 + x^5 + x^0)

Common start values are 0 or 0xffff.

I've got a "known to be good" Delphi implementation for those two.

A good read-up is : http://en.wikipedia.org/wiki/Cyclic_redundancy_check

_________________
Jan
Back to top
View user's profile Send private message Visit poster's website
jamos



Joined: Jun 01, 2004
Posts: 483
Location: Eugene, Oregon, USA
Audio files: 4
G2 patch files: 36

PostPosted: Fri Mar 17, 2006 5:59 pm    Post subject: Reply with quote  Mark this post and the followings unread

I just got it!

It's a CRC 16 algorithm, starts at the first byte after the header. I'll attach my test code here. I tested it on about 8 files, works for all.


G2ChecksumCalculator.cpp
 Description:
A very simple program to calculate the checksum of a G2 .pch2 file. Don't complain about code quality.

Download
 Filename:  G2ChecksumCalculator.cpp
 Filesize:  1.95 KB
 Downloaded:  214 Time(s)

Back to top
View user's profile Send private message Send e-mail Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic Moderators: Nord Modular Editors
Page 1 of 4 [78 Posts]
View unread posts
View new posts in the last week
Goto page: 1, 2, 3, 4 Next
Mark the topic unread :: View previous topic :: View next topic
 Forum index » Clavia Nord Modular » Nord Modular G2 Discussion
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum
e-m mkii

Please support our site. If you click through and buy from
our affiliate partners, we earn a small commission.


Forum with support of Syndicator RSS
Powered by phpBB © 2001, 2005 phpBB Group
Copyright © 2003 through 2009 by electro-music.com - Conditions Of Use