Any software engineers or programming types out there?

[JehuJava]

Not really sure if this is the right forum, but OT didn't seem as applicable.

Basically I want to make a patch/preset editor for my own use but no idea how to go about it. The pedal, that I'd rather not speak aloud, has the ability to import and export presets. But when I open the preset file in a hex editor it makes no sense to me. One file has all the preset data. Could someone point me in the right direction?

I have some VB experience, and I'm not afraid to spend the time learning a new programming language or whatever it takes really. I have a graphic design degree, so you can imagine I like pleasant GUI's vice crappy LCD menus.

I'm not trying to break the law or reverse engineer proprietary information. I just want to make editing presets on a very complicated pedal much easier so I can have some fun with it.

[MontzterMash]

Hi,

First step is going to be to figure out what the structure of that preset file is. That's going to be the biggest factor in the difficulty of this. Since you mentioned a hex editor, I assume it's a binary format of some sort.

The general ways to do that are:

1. Google first, and see if someone else has already reverse-engineered the file format. Could be someone here knows too.

2. Get a variety of preset files that vary in an organized way, and start looking in them and figure out what every byte does. That's a bit like learning music by ear - the more you know a lot about how people encode things in binary file formats the quicker you can recognize the patterns and figure it all out. First time you try it, it can take a while, and be quite daunting, so be prepared for a lot of trial and error. Positive way to look at it - much of that trialing involves playing with your bass through the pedal to see what things sound like.

After that, you can figure out what languages or libraries you'll need to use. Most likely, VB will be fine if that's what you like using.

[JehuJava]

Good idea. I can set all params in preset 1 to 0 so to speak. Then adjust param #1 to 1 then 2 then 3 etc saving each version as preset 2, 3, 4 etc. pretty much running through all the permutations watching how the code cahanges. This should show how the coding works. I imagine after I do a few parameters I won't have to go through each value for each parameter...probably 0, middle, and max.

[bassbrad]

Originally Posted by JehuJava

The pedal, that I'd rather not speak aloud,

There is no shame here we are all effects addicts.

[megafiddle]

You will probably only have to figure out the coding system for one preset.
The rest of the presets will likely use the same format and appear as a sequence
of data blocks, one block per preset. Once you have one, a few quick tests might
identify the others.

You might also see header information at the beginning of the file which identifies it
and never changes.

Also you might see a byte or two at the end that always changes along with any other
byte. This would be a checksum and you would have to determine how it's generated
and generate it each time the file is edited.

[drewfx]

You might search on midi sysex programming. Even if your gear uses USB (or whatever) instead of midi to transfer data, the principles are the same. And the data may be structured similarly if the manufacturer used midi for earlier products.

First thing is to see if the manufacturer will provide you with documentation for the file format (if it's not already available). If not, you have to figure it out the hard way - change something and see what changes in the data. It's also likely that there may be checksums at the end of the data and sometimes every x patches. If you have alphanumeric patch names in the data, it's easy to see how many bytes each patch is by looking for that.

I used to use VBA to load a sysex file into an Excel spreadsheet to try and figure out what was going on. You want to be able to convert from hex to decimal to match most of the settings to what you see on the HW. But sometimes they spread data across more than one byte or pack more than one parameter into a single byte, so be aware of that.

Also note that some devices don't like it if you send them data they don't understand. I've had stuff lock up on me and had to open it up and disconnect the little battery to reset things.

[JehuJava]

Woof. The two things I was afraid of. Check sum and locking it up.

I suppose I can, once I figure out the coding, compare what my program generates versus what the pedal generates for the same settings.

This may be a bigger bite than I was expecting. But let's try it.

[topo morto]

Originally Posted by drewfx

First thing is to see if the manufacturer will provide you with documentation for the file format (if it's not already available)

Definitely worth a try - suspect it would be down to the size of the company...

[DeltaPhoenix]

Crappy LCD screen and it is a complex pedal....I know which pedal this is....

[JehuJava]

Originally Posted by topo morto

Definitely worth a try - suspect it would be down to the size of the company...

ok. i'll give this a shot. I guess it can't hurt. Maybe it'll work.

Originally Posted by DeltaPhoenix

Crappy LCD screen and it is a complex pedal....I know which pedal this is....

Yes you do!

[megafiddle]

Checksums are usually easy to figure out once you've identified which byte is the checksum.

In one system, the checkum is nothing more than the lower byte of the sum of all the other data bytes
(the carry over into the next most significant byte is discarded).

Another system generates a checksum byte such that when all the data bytes plus the
checksum are added, the result is zero (again discarding all but the least significant byte).

You will have to be pretty thorough and careful to avoid lockups. But it can be done.

How many parameters and presets do you have to deal with?

[JehuJava]

[quote=megafiddle;
How many parameters and presets do you have to deal with?[/quote]

46 Parameters in each preset. 50 Presets (though only 30 are created). 1 Parameter can't be changed, so I suppose it's really 45 parameters.

The preset length in the hex editor looks like 64 bytes. Names of the presets are 24 letters long, so it looks like 40 bytes are dedicated to parameter values. This seems troubling...

There are also global settings, which I don't think are stored in the file.

I opened the file in 010 Editor for Mac. I identified the 64 bytes of a preset. I did this by looking at the last preset and saw how it ended (the all end the same) and worked backwards to where the second to last preset ended. I confirmed this all the way back to the beginning. I found the first 16 bytes of the file seem to be different than the presets; looks like it indicates that the file is a preset file.

So, for the sake of comparison, I deleted the first 16 bytes of the file and lengthened the lines in the editor to 64 bytes. all the presets lined up beautifully on top of each other. The big tip off was the preset names all began at the same point.

I have an Excel spreadsheet of all the presets with all the parameters listed. I'm not seeing any connection between the parameters in the sheet with the coded bytes. For instance, parameter 1 is the same for 27 of the 30 presets only 3 have different values. This should have been easily seen when looking at the preset file now that they are perfectly on top of one another. But it's not.

I TAKE THAT BACK...just found parameter 1. I'll keep comparing and see where I get.

[NKUSigEp]

Originally Posted by bassbrad

There is no shame here we are all effects addicts.

Perhaps it is a 5th^/Oct^ setting on some sort of a pitch shifter and he knows that if it got out, this pedal would cause the destruction of the world and possibly the entire universe.

[JehuJava]

Originally Posted by NKUSigEp

Perhaps it is a 5th^/Oct^ setting on some sort of a pitch shifter and he knows that if it got out, this pedal would cause the destruction of the world and possibly the entire universe.

I wish. The builder sometimes shows up here and was hoping not to cause problems. I just wanna make editing easier, albeit offline and not real time. But it can help me get a basic starting point for a preset faster I think.

15 parameters identified. The remaining are kind of cryptic. And since there are 25 bytes still needing identifying but 30 parameters left, could 1 byte control more than one parameter?

[bongomania]

It may be that some group of parameters are enabled or disabled by one bit, so they could be excluded from some strings. E.g. turning an EQ off might mean the control bits for each band are not included. Just a guess.

[JehuJava]

Originally Posted by bongomania

It may be that some group of parameters are enabled or disabled by one bit, so they could be excluded from some strings. E.g. turning an EQ off might mean the control bits for each band are not included. Just a guess.

You may be right. Once I narrow down the last few I should be able to see a pattern. Right now there's still a few too many bytes unknown to determine which are which.

So, an observation that I'm not ready to tackle yet but will have to when I'm ready to program my own presets: I can't tell if their using signed or unsigned bytes. The parameters seem to jump back and forth between the two. Maybe I need to look at each parameter on its own? For instance, one preset has the parameter of 56 in the pedal and so does another preset. When you look at the preset file one string will show a signed value of 56 and the other will show -200. Both are essentially showing 56.

Another observation is a parameter that is non-numerical, like on or off or mute, is showing up in the preset file as a range. Like 0-6 = on and 7-16 = off. Weird range choices. Or a value of 50 in the pedal shows up in the preset file as maybe 47-53. Decimal rounding? Range? I will deal with this later.

[JehuJava]

So interesting...!

[topo morto]

Originally Posted by JehuJava

When you look at the preset file one string will show a signed value of 56 and the other will show -200. Both are essentially showing 56.

In a 16 bit 2's complement representation, I think those would have the same 2nd byte...

56: 00000000 00111000
-200: 11111111 00111000

so could it be that the 56 is actually only an 8-bit value, and the first byte relates to something else?

[megafiddle]

How are you looking at the data? As bytes? Or 16 bit words?
And as hex values or decimal?

I am confused about the -200; it doesnt fit in one byte which has a range of -128 to 127.

Does the file end right at the last preset? And are data bytes changing only one at a time
with a single parameter change? If so, sounds like there are no checksums.

Sounds like you're making progress. Interesting project. I wrote a patch librarian for my
DX7s keyboard long ago. For Atari 520ST in GFA compliled BASIC. I had the advantage
of a documented file structure though.
With the PC and Windows, I do everything now in C.

[JehuJava]

Originally Posted by megafiddle

How are you looking at the data? As bytes? Or 16 bit words?
And as hex values or decimal?

I think as bytes. Although, I'm using the character representations to do this. This was how I was able to line up the presets in rows to compare characters, by aligning the preset name. Then I looked for the same changes amongst presets by the characters (and the values if to verify). I found the easy ones. But it looks like the rest as combinations and strings. I'm trying to understand what this means by searching the internet.

Originally Posted by megafiddle

I am confused about the -200; it doesnt fit in one byte which has a range of -128 to 127.

I think I got this backwards. I was typing from memory.

Originally Posted by megafiddle

Does the file end right at the last preset? And are data bytes changing only one at a time
with a single parameter change? If so, sounds like there are no checksums.

Yes. That's how I determined where all the presets end.

Originally Posted by megafiddle

Sounds like you're making progress. Interesting project. I wrote a patch librarian for my
DX7s keyboard long ago. For Atari 520ST in GFA compliled BASIC. I had the advantage
of a documented file structure though.
With the PC and Windows, I do everything now in C.

Making a little bit of progress. I'm basically learning in reverse.