Reactions and preorder status

It’s now been almost a month since I published the long-awaited post about OpenDeck v1.0. To say that I received overwhelming amount of responses would be an understatement. So far, OpenDeck has been featured on Synthtopia, AskAudio, GearNews, AudioZine and many other places I forgot to mention. A sincere thank you goes to every supporter out there!

The big news here is that I received enough pre-orders for OpenDeck boards. After all payments are done, I’ll order a new batch of boards and send them once they arrive. This process should take about a month, hopefully less. Note that boards can still be pre-ordered, but next batch of boards will be ordered a bit later.

I have also opened my own domain (finally!), so web configuration utility can now be found on shanteacontrols.com. This blog will also move to that domain soon, so that everything is in one place.

This is all for now, stay tuned for more!

OpenDeck platform v1.0

You’d be forgiven for thinking that OpenDeck project is dead, as well as this blog – after all, it’s been more than a year since the last update. While it’s true that in the past year I haven’t had much time to work on the platform, last couple of months have been full of development again, and this post is an announcement of OpenDeck v1.0 – stable version of firmware, hardware and Web configuration utility. This time, I’ll concentrate on user-facing changes only – low-level changes are going to be discussed in future posts.

In case this is your first visit here, I’ll copy OpenDeck info straight from GitHub:

OpenDeck is a platform suited both for prototyping and developing custom MIDI controllers compatible with any MIDI software on any OS. Main part of the platform is board on which various components used to build a MIDI controller can be connected. The board supports the following components:

  • Buttons
  • Encoders
  • LEDs (single color or RGB)
  • Potentiometers
  • FSRs (force-sensitive resistors)

Hardware

Since my last post, OpenDeck board went through two revisions. I’ve fixed some stuff, and added some as well, however, the basic layout is exactly the same, as well as dimensions. Final version of the board is black – it sure does look nicer compared to ugly green one.

2017-01-30-16-51-26

Now there are three indicator LEDs, compared to two on last revisions. One is for power, and other two are active when there’s MIDI traffic on board (one for input and one for output). Those two LEDs serve double-purpose, as they’re both on while in bootloader mode.

On MIDI side of things, I’ve replaced 6N138 optocoupler with way-faster 6N137.

Configuration utility

This is the big news. Really big. Ever since Google announced the support for Web MIDI in Google Chrome, I wanted OpenDeck configuration utility to run inside it. Building one app which runs on any platform is definitely better than coding three different apps for Mac, Linux and Windows. Since my knowledge of web is pretty miserable, I’ve hired two guys (programmer and designer) to design and build configuration utility based on my specs. I don’t know much about low-level details of utility, but I do know it involves some magic words such as “Angular” and “Javascript”. Concept of utility is actually pretty simple – it has sidebar containing configuration blocks available on OpenDeck – MIDI, buttons, LEDs, encoders and analog stuff. It also has activity log, displaying all incoming and outgoing messages, info window and backup and restore functionality. All configuration is done using custom SysEx protocol explained here. Below are the screenshots.

screenshot859
MIDI configuration block

On MIDI screen, user can configure couple of settings related to MIDI protocol itself, as well as channels for MIDI messages used on OpenDeck. Toggling DIN MIDI in to USD MIDI out will result in all MIDI traffic received via DIN MIDI in connector being forwarded to USB MIDI out, so any older MIDI gear can be used as USB MIDI controllers using OpenDeck  board. You can read more about running status and note off messages here.

screenshot860
Button configuration block

Button window lists all buttons which are possible to configure on board. Even though board has 64 inputs for buttons, 32 analog inputs can be converted to digital inputs (that is, buttons), resulting in total of 96 possible buttons. One of the cool features of utility is that pressing the button connected to board will automatically blink the correct button ID in utility, so that you don’t have to waste time finding it. You can read about how is that accomplished here. Another cool thing is that number of buttons (or anything else) isn’t hardcoded, but generated by the utility. Before the configuration section is rendered, a request is sent to board asking it how many buttons are possible to configure, so utility is future-proof if I’m ever going to increase or decrease number of buttons, encoders etc.

screenshot861
Single button configuration

Clicking on certain button brings up modal window which lists all possible parameters for single button. Here, user can configure button as momentary or latching. MIDI message sent by button can be chosen as well (note or program change) and also, any MIDI ID (0-127) can be assigned to specified button. MIDI ID is used as note or program, depending on which message type is selected.

screenshot862
LED configuration block

LED configuration lists some global parameters applied to all LEDs, as well as list of all LEDs which are possible to configure. OpenDeck supports 48 single-color LEDs, or 16 RGB LEDs, since RGB LED is actually three LEDs in single case.

screenshot863
Single LED configuration

Any LED can have assigned any activation note (0-127). When local control is disabled, LED is controlled with received MIDI note and velocity. You can read more about LED control here. When enabled, any button which sends note same as activation note will control the LED instead. LED can also be configured as RGB LED. Last two options are used for testing LED states (when RGB LED is enabled, different colors can be picked from state test field).

screenshot864
Encoder configuration block

Encoder section displays total of 32 encoders. 64 digital inputs available on OpenDeck can be used to connect those 32 encoders (one encoder takes 2 digital inputs). Note that analog inputs cannot be used to connect encoders.

screenshot868
Single encoder configuration

Encoders need to be specifically enabled in order to function. Encoders send CC messages, and any CC number can be assigned to specific encoder (0-127). Two encoding modes are available: 7Fh01h and 3Fh41h, which can also be inverted.

screenshot865
Analog configuration block

Analog configuration section displays total of 32 components.

screenshot866
Single analog component configuration

Analog components need to be enabled in order to function. This is to avoid junk coming out of floating analog input when nothing is connected to it. All analog inputs can be configured as potentiometers, FSRs and buttons. Default is potentiometer. Any MIDI ID can be assigned (0-127). Potentiometer type will send CC messages with CC number specified as MIDI ID, FSR type will send MIDI note event with note being MIDI ID, and button type will send note event specified in button configuration section. Invert option applies only when potentiometer is active type. Lower and upper CC limits can be used to scale CC values coming from potentiometer and FSR.

screenshot867
Info window

Info window contains some basic info about the board (firmware and hardware version). Checking for updates is done by comparing last tag on OpenDeck GitHub to firmware version currently present on board, which is a very cool feature. Other options include rebooting the board, factory reset and bootloader mode, which is used for firmware update process.

I can say I’m very proud of configuration utility, as it’s one of the first WebMIDI apps used for configuration using SysEx messages! Building a MIDI controller sure does seem easy now. I haven’t had the chance of testing it on Linux distros yet, but it works (mostly) flawlessly on Windows and Mac OS X – mostly because of WebMIDI still being finicky. For instance, MIDI connection sometimes won’t be detected, and leaving the utility idle for some time can sometimes lead to issues. On top of that, Google also requires SSL connection when using SysEx messages via WebMIDI which is a real PITA. Really Google? I hope those issues will be resolved one day, but for now, I can live with it. I also hope other Web browsers will include support for WebMIDI soon – not everyone likes Chrome. There is some activity with WebMIDI implementation in Firefox, though.

Documentation

Undocumented code is worthless. I’ve written extensive wiki on OpenDeck GitHub, so you can check entire documentation here – everything from schematics, connection diagrams, SysEx protocol, firmware update process etc.

Open Source

Most of OpenDeck really is open. I’ve done my best to write clean and efficient code and document entire project. Exceptions to this are board design (gerber files) and Web utility. Those two things are truly custom solutions, and OpenDeck is a platform open enough to be used without them, since my GitHub contains complete schematics and full documentation on SysEx protocol. Maybe someone will build a better board and better configuration app – who knows? In the meantime, my board and utility really are the only incentives for someone to actually spend any money on OpenDeck. At the end of the day, whenever someone buys it, it will make me invest more time into platform, which translates to less bugs, more features and possibly new products. So far, development of platform has been pretty slow. Supporting it will definitely change that.

Availability

Now that board, firmware and configuration utility are ready, you can contact me for pre-order. Pre-order here means that I’ll send the boards to manufacturing once I receive at least 10 pre-orders. I’m selling the board for 111€. Package contains:

  1. Board
  2. USB cable
  3. MIDI cable
  4. Access to online web utility
  5. One year guarantee

Bugs and feature requests can always be reported here.

Thanks for reading!

Marching forward with OpenDeck: The hardware

This post will shed some light on the development of OpenDeck platform, since it’s been quite some time I’ve done any updates on this project. Don’t let the silence fool you – project is very active.

For those who don’t know what OpenDeck is (there are couple of posts about it on this site) – it’s a MIDI platform designed to relieve you of doing tedious stuff in the build-process of a MIDI controller. Those who want to build their own controllers quickly find out that it’s not really a small thing to accomplish, even if there are some great platforms out there which can make the job much easier, such as Teensy or Arduino. You need to know how to write code, choose the right hardware, install software development tools on your computer, deal with all kinds of errors (both hardware and software) etc. OpenDeck allows you to forget about all of that. It’s a system which allows you to pick basic components of your MIDI controller – such as potentiometers, buttons, encoders, LEDs etc. – plug them into the OpenDeck board, and actually use them in your MIDI controller in a matter of minutes. Neat, right? So, how does it work? I’m going to talk about it in a couple of posts, since the project has several layers. In this one, I’ll try to do a hardware architecture breakdown of the project.

The board

The board is 10x10cm in size. It’s 1.6mm thick and the current color is green. It has four M2 mounting holes, one in each corner. I don’t have pictures of the board since I’m currently waiting for it to arrive from ITead studio, but here’s render of front and back (the holes in the corners are not displayed):

eagleUp_OpenDeck-r1_board_top

eagleUp_OpenDeck-r1_board_bottom

Specifications

You can connect various components which are the building blocks of your MIDI controller into OpenDeck board. It allows you to connect:

  1. 64 buttons or 32 encoders max (one encoder is kinda like having two buttons)
  2. 32 potentiometers or FSRs (force-sensitive resistors)  – support for other analog sensors is planned as well
  3. 48 single color or 16 RGB LEDs

The board also contains DIN MIDI in and out ports. MIDI in port can be used in two ways:

  1. MIDI to USB converter, allowing you to receive data from MIDI gear and translate it to USB MIDI to control your MIDI software on a computer
  2. LED control, allowing you to receive data from MIDI gear and control LEDs connected to OpenDeck board.

Schematic

Complete schematic and PCB files can be found on this link.

Power

The board has only one power connector – USB (B type). Board can be used either using computer or standalone, in which case USB wall charger is required – any one will do, as long as it meets USB power spec (5V/500mA).

ESD protection

USB data and voltage lines have ESD protection diodes. There’s a nice article on electro-static discharge on Wikipedia.

Indicator LEDs

There are two LEDs on board – power and bootloader LEDs. Power LED is constantly on when board is powered on, while bootloader LED is only active when firmware update is in progress.

Microcontroller

The brain of OpenDeck board is same microcontroller used in Arduino Leonardo and Teensy 2.0 – Atmel ATmega 32u4. That little microcontroller has USB hardware built right into the chip which allows your computer to recognize it as generic MIDI device, so that there are no drivers required on any major platform (Linux, Mac OS X, Windows).

Button/encoder hardware

In order to read buttons and encoders, a matrix setup is used. There are 8 input rows controlled by 74HC165 shift register and 8 columns controlled by 74HC238 decoder. Matrix uses column-switching to read data and there is only one active column at time during which all 8 rows are being read. There is more info on how button (and LED) matrix works on this link.

LED hardware

LED setup also uses matrix. It has 6 rows and 8 columns. Driving LED matrix directly from microcontroller is generally un-advisable, since the microcontroller isn’t really meant to be current source (or sink) so its capabilities are quite limited in that regard. Matrix can draw a lot of current, so if not careful, you can easily damage the microcontroller. In order to solve this, several components are used:

  1. LED hardware uses dedicated voltage regulator, LP2985-N. Regulator outputs stable 4V.
  2. Each LED row has a MOSFET transistor which turns the row on or off, triggered by a state of digital pin on microcontroller. This way, very little current is required to turn the MOSFET on, and when it is on, transistor draws the current from 4V voltage regulator instead of microcontroller, and feeds it into LED.
  3. Each column is connected to ULN2803 current sink, since 6 rows of LEDs can generate about 200mA of current and absolute maximum microcontroller can handle is 40mA.
  4. Current sink inputs are connected to another 74HC238 decoder so that only 3 pins are required to handle column switching instead of 8.

The big question here was which resistors to choose for LED rows. In order to achieve full brightness, most LEDs require 20mA of current. In matrix, LEDs get switched on and off very fast without you noticing it. Current flawing through LEDs in matrix is called “pulse” current, since it flaws for a very short period of time. If you would pick a resistor for 20mA of current in LED matrix, you would notice that LED isn’t really set to maximum brightness, since the current is actually lower. Because of this, we will calculate the value of resistor to achieve 30mA of current. Another problem rises here: each color of LED has different voltage drop. For instance, red LED usually has voltage drop around 1.7V. Green has around 2.2V. In order to select resistor value, I took the average voltage drop for most popular colors, based on this table. The average drop is 2.3V. For 30mA current, 4V voltage source and 2.3V of voltage drop, a resistor value of 56 ohms is chosen. Since this calculation is based on average voltage drop, the current will vary depending on chosen color.

Analog hardware

For analog components to work reliably, stable voltage is absolutely required. Like LED hardware, analog part of the board also has dedicated 3.3V voltage regulator, MCP1825. To read values from 32 analog components, two 4067 analog multiplexers are used.

MIDI hardware

MIDI in hardware requires the use of opto-coupler – I’ve selected 6N138 as recommended on MIDIbox forums and official MIDI documentation. MIDI out circuit is the same as recommended on Teensy web site.

That would be all in this post – in the next one I’ll talk about the software running on the board.

Building Tannin 2

In this post, I’ll be talking about build process of my latest and greatest MIDI controller; Tannin 2. It took me almost 5 months to actually finish it since there were so many architectural changes in it that I basically started from scratch.

The design

Tannin 2 design is, naturally, very similar to the original Tannin controller (incidentally, I never actually wrote a post about that one since it was finished some months ago before I started this blog). Since Tannin 1 is first Shantea Controls controller ever built, it has a special meaning for me – not so much as a physical product since it’s disassembled (I used some of its parts for Tannin 2) – but more as an idea. I had to make sure that Tannin 2 would be a worthy successor. For the reference, here’s the rendering of original Tannin controller:

Tannin

Assembled controller:

IMG_8520

And one from the back:

100_2743

I can assure you it looks much better on the pictures than it really looked.

It had 16 potentiometers and 16 buttons in 4×4 grid, 3 additional metal buttons and 4 LEDs at the bottom. Very simple and very efficient layout. Over the time, I got used to it so much that I mapped virtually every function I need in my DJ software (Traktor Pro) using nothing but this controller. Still, it was my first controller, and naturally it had some flaws. First and most obvious one, there were too many screws. Second, that box was huge, too heavy and after few months it started to crack all over the place. It was also such a pain in the ass to carry around. On the hardware side of things, it featured Arduino Nano and circuitry that later became the basis for OpenDeck board. If you’re wondering what’s that chemical – it’s Flavan-3-ol.

So, now that we have Tannin 1 as a reference, let’s move on to Tannin 2.

Tannin 2 was imagined to be an improvement in every area – not just the flawed ones. When I thought about the design, I was trying to figure out what was missing in original controller. First of all, I wanted better buttons than those tactile ones and more LEDs. I also wanted either faders or encoders. Eventually I discarded the faders idea since it was impossible to fit them in same dimensions (I wanted to keep plate dimensions exactly the same – 200x300mm). So, after some consideration, this is what I came up with:

Tannin2

As you can see, design is really similar to original Tannin, but with few tweaks. Below 16 potentiometers, I wanted to have 4 encoders, and under them 16 silicone buttons with white LED inside each of them. 4 LEDs at the bottom got removed since a) I didn’t have much room left, and b) who needs 4 LEDs at the bottom when you have one in each button above? Potentiometers and metal buttons remained the same. I also added number 2 to Tannin name, even though it actually looks like letter ‘R’. Tannin Remixed? Acceptable as well. Notice that there are only few screws in this one. All in all, I was happy enough with that layout to proceed to actual build process.

Plate

As with the Ceylon controller, I wanted to experiment a bit with plate color. This time, I picked green one, and it really looks great – same as any gravoply plate.

tannin_plate

The case

Until I built this controller, cases were always source of biggest frustration, since – let’s face it – they looked like crap, and thus ruining the overall feel of entire controller. Well, I didn’t want to go through same story again, so for a couple of weeks I was searching for someone who would do the job right. Whether or not I was successful, judge for yourself:

case_front

And back.

case_back

Finally, I have gorgeous case! That green plate really blended well with the tone of the wood. Also, its weight is just about right, not too light (it’s wood after all), and actually not heavy at all. Perfect!

One more thing – I redesigned USB cable plate:

tannin_usb_plate

Electronics

For second revision of Tannin controller, I decided to use PCBs crafted specifically for Tannin 2, instead of using OpenDeck reference board. This proved to be a very challenging task, as I literally had to do everything from scratch. The main premise behind designing electronics for this one was that I was going to use ATmega32u4 as microcontroller. In the past, I used either Arduino Nano or Arduino Pro Mini. Both of those boards are fine on their own, however, the biggest drawback to using them is that they have no native USB capabilities. In practice, this meant that I had to use serial-to-usb adapter to program them and couple more layers of software to make them talk MIDI. In my commercial designs I also used those boards, but in addition to main board I used USB MIDI board which translated serial signals from Arduino to USB MIDI. Works just fine, but that USB MIDI board doesn’t support MIDI System Exclusive (primary method of configuring the controller) and it represents additional hardware. So, using ATmega32u4 solves both of those problems since it has USB built right into the chip. In case you didn’t know, Arduino Leonardo, Arduino Pro Micro and Teensy 2.0 use that exact microcontroller, however, this time I needed my own custom board. I needed it to be as small as possible – so Leonardo was discarded right away, I needed pretty much every available pin – discarded Pro Micro at this point as well and I also needed ability to use a custom bootloader for uploading new firmware to board. Now, you absolutely can put a new bootloader on a Teensy board – but you can’t upload the new code on it using Teensy loader then since that bootloader is proprietary, an aspect of Teensy I’m not particularly fond with. So, when you put new bootloader on Teensy, what’s the point of buying Teensy in the first place? Also, it was cheaper to buy pure ATmega32u4 chip with few external components than buying Teensy. I’ve been delaying this design with each controller since I figured that soldering SMD is going to be much harder than soldering standard DIP packages. I was right, but just because it’s harder doesn’t mean it’s impossible. You just need steady hand and some patience. The final result was this:

tannin_main_pcb

Only 5x5cm! Obviously, main component is ATmega32u4, the mighty brain which does all of the processing. Next, it features my standard hardware for LED/button matrix, 74HC238 decoder and ULN2803 current sink. For reading button rows, this time I added 74HC165 shift register. I had few 4021 registers lying around, but 165 register can work a bit faster, so that’s why I’ve chosen it. I also added reset button, ICSP header so that I could program it with external programmer (USBasp) and also MIDI header, which is nothing more than +5V, GND and RX/TX serial pins from microcontroller. Having them on header allowed me to test hardware MIDI on this board as well, and it works! This means I can expand this board with real DIN MIDI in/out connectors. Not that I have any use for it at the moment, but since this board will serve as a basis for OpenDeck v2 reference board, I wanted to see how it’s going to behave. Board also features pin headers on the edges to connect potentiometers, buttons, LEDs and encoders. One resistor is missing in this picture – at that point I was unsure what to do with HWBE (hardware boot enable) pin on ATmega, but more on that later.

Now, having the main board finished didn’t mean I was done with electronics. At all. First of all, those silicone buttons needed a PCB. Encoders also needed a PCB, and potentiometers as well. That’s lots of PCBs. I ended up designing a single 10x10cm board with panelized designs for each part. In total, I needed 4 separate PCBs for buttons, 2 boards for encoders and 2 boards for potentiometers. I also needed a small board for USB connector which would be housed in the box. Luckily, I managed to fit all of that in a single board, and the guy who cuts gravoply plates for me also did the cutting of PCBs.

tannin_components_board_single

To make one controller, I actually needed two of those boards, however, you can’t order less than 10 pieces, so that wasn’t a problem.

tannin_components_board_cut

Free candy! I mean PCBs!

This is how Tannin 2 looked inside after first round of soldering:

tannin_pcbs

Analog boards feature one 4051 multiplexer each. One of them is “slave”, other is “master”. This only means that slave (right one) connects to master (left one) using 6-pin IDC cable. That allowed me to use only 7 wires from master analog board to main board. Encoder boards feature 5 pins each. Both boards also connect to main board. Four boards with LEDs and silicone buttons have shared columns, so each board connects to the next using four wires. Finally, those four wires are connected to main board. Boards also have one LED/button row wire each, so that’s two additional wires from each of the four boards to main board. Three metal buttons below also connect to main board on assigned pins.

tannin_buttons_pcb

I used some hot glue to ensure the wires don’t move and break. Each board has one resistor for LEDs and 4 diodes for button matrix (there are actually 2 diodes per package so there are 2 packages per board). Diodes for three metal buttons are placed on main board.

Since encoders are nothing more than a two buttons, they are connected in separate button matrix. Reason for separation is that button/LED matrix switches columns every 1ms, and that’s too slow to read encoders properly. Setting the matrix to faster switching rate would result in LED flickering since I also needed PWM for LEDs.

Final look inside:

tannin_inside_finished

That wooden part makes sure the button PCBs don’t bend. Also, main board is designed so that it fits perfectly on four screws holding two PCBs together. Finally, I added some hot glue to secure all contacts:

tannin_inside_finished_2

Entire schematic will be available on my GitHub at some later date.

Software

As I mentioned before, Teensy 2.0 board uses ATmega32u4, same microcontroller that I picked. Even though Teensy boards are compatible with Arduino, software is of much higher quality and they can also be programmed as USB MIDI devices, something Arduino boards aren’t officially capable of doing (at least in Arduino environment). Selecting Teensy 2.0 board in Arduino IDE and then trying to upload it to Arduino Pro Micro or Leonardo won’t work because Teensy uses Teensy Loader to load software on board, which in turn communicates with their HalfKay bootloader, which you can’t download. However, I needed that USB MIDI stack that Teensy boards provide. So, I simply connected my main board to USBasp programmer, selected Teensy 2.0 as a board, USB MIDI as device type and uploaded it to board. Note: if you want to upload code to Teensy/Arduino using external programmer, simply hold shift and then press upload. That worked flawlessly! Board immediately showed up as Teensy MIDI on my computer. Awesome! Now, the hard part was moving all of my existing OpenDeck code to Arduino-style format. In the process, I integrated most of Ownduino functions directly into OpenDeck library. I could talk about the software part for hours, but you can easily check the code yourself on my GitHub. This was all very nice, but every time I needed to change something in the code, I needed to connect external programmer to main board. In practice, this meant constant opening and closing of controller – quite tedious. I needed a bootloader and I found one. It’s called USB Mass Storage Bootloader and it’s really fantastic. After you reboot your board into bootloader mode, your PC shows a new device connected to your PC: removable drive! It shows two files: EEPROM.bin – which contains all parameteres stored into EEPROM (obviously) and FLASH.bin – the compiled code. To upload new code, simply delete FLASH.bin, copy new one, eject the drive (very important – ejecting it actually triggers the upload process), and then simply reconnect controller to USB.

tannin_bootloader

The trouble I had with bootloader was that the controller was constantly turning on in bootloader mode – it never showed as MIDI device. Later I found out I need to change certain fuses on microcontroller. USB AVR microcontrollers generally have two fuses responsible for deciding whether to run user program or bootloader: BOOTRST and HWBE. If you enable BOOTRST, microcontroller is going to run bootloader everytime it’s reset. HWBE, on the other hand, acts a bit differently. If you enable HWBE and disable BOOTRST, your device will reboot into bootloader only if HWBE is shorted to ground before you turn on microcontroller. This is why you need to hold reset key on certain devices before you can update firmware on them. I figured I don’t really need any of these fuses, so I disabled both of them. I want microcontroller to always run application program, and bootloader only on demand. To trigger the bootloader reset, I need to send specified MIDI System Exclusive message to controller. After that, controller reboots into bootloader mode, showing as a disk drive. Just what I want. Great! Another puzzle was how to generate that FLASH.BIN file? I only use Arduino IDE for testing out stuff and use Atmel Studio for any bigger project. After you compile project in Atmel Studio, it generates .hex, .elf and few other files inside your project directory. You can also define “external tools” in Atmel Studio. I defined one such tool to convert .elf to .bin. Basically, you need avr-objcopy tool with these parameters:

-O binary -R .eeprom -R .fuse -R .lock -R .signature <compiled_project>.elf FLASH.bin

I only need to figure out how to do that automatically after compilation so that I don’t have to manually invoke the command.

Conclusion

This is how controller looks right now – fully assembled:

tannin_assembled

So, here’s a quick video showing Tannin 2 in action. Notice how LEDs fade now – I finally added PWM support with selectable fading speed using System Exclusive.

Building Ceylon

This is a long overdue post about Ceylon, another Shantea Controls controller, only this time, one that I’ve built for myself. Of course, it’s based on OpenDeck platform.

Ceylon was actually built about three months ago. Reason why I haven’t written about it is a combination of things really. First, it was in a beta period for more than a month, and after that I just didn’t have much time to write posts, as I was really busy (and still am) with other projects that I’ll hopefully write about soon.

Name and design

Let’s start with a name. After Anandamidi and Sensimidia, it was time to bring back tea in Shantea Controls again. So, Ceylon is actually well-known high-quality tea originating from Sri Lanka, former British colony known as Ceylon (hence the name). There are couple of variations of it, black one being my favorite (with some spices of course, that is, loads of ginger).

10678704_306345016225412_2880265286737813749_n

Picture above is a rendered drawing of Ceylon design. Numbers around the circle are coordinates of Kandy District, heartland of tea production in Sri Lanka. So there you have it, a Ceylon story!

Layout

Since Tannin was my only controller so far, I’ve mapped nearly every function I need, so when I started designing my second controller, I wanted something really simple that would complement Tannin well. There was no need for a complicated layout or redundant functions, that’s why its design is really minimal. There are 3 faders, 12 orange LEDs for some eye-candy, err, I mean VU-meter, shift button, two potentiometers above faders for gain control, six anti-vandal switches with blue LEDs, and disk. Most significant change here from Tannin is the use of encoder. Well, not your usual encoder though. It’s a salvaged HDD motor acting to be encoder actually.

Plate

Ceylon is first controller I’ve built which doesn’t use standard black/white Gravoply plate, as I grew tired of it. Instead, I’ve picked blue plate this time, and it looks really gorgeous.

WP_20150224_028

I also redesigned USB plate.

WP_20150224_017

Case is white again, just like Tannin.

The disk

As I’ve already stated, only thing new in Ceylon is the disk. So, how does it work? The motor in mechanical hard disk generates phase-shifted sinusoidal waveforms when spinned. The faster you spin it, larger the amplitude and frequency of waveforms. By examing waveforms, you can determine disk direction, which is what I needed. There are couple of issues with this:

1) Signal amplitude is way too low to be sampled directly by a microcontroller, unless you spin the disk real fast, which is kind of missing the point of whole setup as you want fine grained control. My measurements showed that disk generates +-500mV when spinned at maximum speed (maximum being somewhat subjective term, I spinned it by hand).

2) For encoders, you don’t actually need analog signals, but digital ones. By examining output data from two encoder pins, you can easily determine its direction. Those two outputs are called quadrature outputs, as they are 90 degrees out of phase.

600px-Quadrature_Diagram.svg

ScreenShot209

Making the signals digital

Basically, what this setup needed was ADC (analog-to-digital converter). For this use, I’ve chosen LM339, a very cheap, available and popular comparator. LM339 contains four comparators in a package, making it suitable for this setup, as I needed only two. Comparator takes two inputs, and simply determines which one is bigger. If voltage on non-inverting input (+) is larger than voltage on inverting input (-), output is digital 1, or Vcc, and if non-inverting input is smaller than inverting input, output is digital 0, or -Vcc (in this case GND). Very simple. But as usual, there are couple of caveats.

Issues

Connecting motor inputs directly to LM339 isn’t such a good idea, for two reasons:

1) What happens if two signals are very close to each other? Comparator would output bunch of ones and zeros very fast, which is actually junk, so you get unreliable results.

2) According to LM339 datasheet, you cannot apply more than -0.3V at either of its inputs. This is a problem, as disk actually outputs about -0.5V when spinned real fast.

Hysteresis

It took me really long to figure out what hysteresis actually is, but it’s actually really simple. Using hysteresis on comparator, you are creating two thresholds for generating two output states, that is, you are setting one threshold for output to be Vcc, and second one to be -Vcc, that is GND. This is achived by applying positive feedback from output back to input, using two resistors. Since signal from motor is really low, I’ve designed hysteresis for low values, just to keep signal from circling around switch point. Hysteresis is calculated using this formula:

Vth+ = -VN ∙ (R1 / R2)
Vth-  = -VP ∙ (R1 / R2)

Vth is threshold voltage, Vn is negative output (in this case 0V), and Vp is positive output (+5V in this case).  I wanted to set positive threshold above 0V, and negative below -5mV, so resistor values are 1k for R1, and 1M for R2:

Vth+ = -0 * (1000/1000000)
Vth+ = 0V
Vth- = -5 * (1000/1000000)
Vth- = -5mV

So, when positive input voltage is above 0V, output is 1, and only when negative input drops below -5mV, output is 0. This way I created “dead zone”, or area where my signal can be of any value (between 0 and -5mV), without affecting the output of comparator.

250px-Smitt_hysteresis_graph.svg

Graph is showing the input signal (U), usage of comparator on that signal without hysteresis (A), and output signal from comparator with applied hysteresis (B).

2000px-Op-Amp_Schmitt_Trigger.svg

Picture above shows hysteresis setup. This is how you debounce your inputs using hysteresis, so that your output never becomes gibberish.

Voltage clipping

Now, there is one more concern. As I stated already, inputs on LM339 cannot be smaller than -0.3V. To accommodate this, I used BAT46 Schottky diode on inputs, having anode connected to ground. When the input is positive, diode doesn’t do anything as current cannot pass through. When the input is negative, diode still won’t do anything, as long as input voltage doesn’t become smaller than -0.3V. Those diodes have a voltage drop of about 0.3V (how convenient), so, when input voltage exceeds -0.3V, current will pass through diode, and voltage on comparator input will actually be voltage drop on that diode, and it will not exceed those -0.3V. Two problems solved, yay!

encoder

Software side of things

Now that I’ve taken care of hardware, it was time to actually read and process signals from motor. For this, I’ve used modified encoder library for Teensy/Arduino. Library is great as it has two really clever parts:

1) Since it’s written with Teensy/Arduino in mind, it automatically detects whether the pins on your microcontroller on which you’ve connected encoder have interrupt capability. If they do, library reads encoder using interrupts. Since HDD motor can be spinned real fast, I’ve connected both motor pins to interrupts (pins 2/3 on Arduino) in order not to miss any pulse. Pins 2 and 3 are two out of four unused pins on my OpenDeck board, so this was very convenient.

2) It has “predicting” algorithm, giving your encoder 4x more resolution. This is achieved by remembering previous state of pins, and comparing it to current state of pins with a lookup table. Lookup table contains valid encoder transitions:

ScreenShot210

This works really, really well, but since HDD motor jumped a bit when changing direction, I’ve implemented additional debouncing:

void OpenDeck::readEncoders(int32_t encoderPosition) {
if (_board == SYS_EX_BOARD_TYPE_OPEN_DECK_1) {
if (encoderPosition != oldPosition) {
if (millis() - lastSpinTime > ENCODER_DEBOUNCE_TIME) initialDebounceCounter = 0;
if (encoderPosition > oldPosition) {
if (!direction) { initialDebounceCounter = 0; direction = true; }
if (initialDebounceCounter >= ENC_STABLE_AFTER) {
sendPotCCDataCallback(127, 127, 5);
} else initialDebounceCounter++;
}
else if (encoderPosition < oldPosition) {
if (direction) { initialDebounceCounter = 0; direction = false; }
 initialDebounceCounter++;
if (initialDebounceCounter >= ENC_STABLE_AFTER) {
sendPotCCDataCallback(127, 1, 5);
} else initialDebounceCounter++;
}
oldPosition = encoderPosition;
 lastSpinTime = millis();
}
}
}

If there is a movement detected, code first checks for whether the disk hasn’t been moved for more than ENCODER_DEBOUNCE_TIME (70mS). If it hasn’t, it resets the debounce counter. This is to avoid extra pulses when disk is slowing down. After that, initialDebounceCounter variable is incremented until it reaches 1. This is to prevent disk jumping when changing direction.

So there you have it, a pretty good resolution from HDD motor for use in MIDI controller! In all honesty, its resolution is far from optical encoders, but as this is more of a proof-of-concept, I’m really satisfied with results.

LEDs on anti-vandal switches

This was another issue that I had while building Ceylon. Anti vandal switches (the ones around the disk) I had around have blue LED on them. Their pins are separated from button pins. Those button pins can be connected in NC and NO configuration, so there’s 5 pins in total. Since I’m using shared column button/LED matrix in my OpenDeck platform, I’ve connected – pin of LED and one of button pins together, going into same column, and then + pin of LED into LED row, and second button pin to button row. Nothing out of ordinary, right? Well, something weird was happening with this setup. Whenever I pressed the button, LED on it lighted up. This is not the behavior that I expected nor wanted, so for a while I had no idea what was happening. Only few days later, I’ve discovered that it doesn’t really matter where you connect + and – of the integrated LED on button, since there are 2 LEDs inside, one of which has anode on + pin, and second one on – pin.

anti vandal configuration

Okay, but why does the LED turn on when button is pressed? For this, answer is in matrix setup. LED/button matrix works by switching columns very rapidly. Only one column is active at the  time, and during that time, it is connected to GND, while others are connected to Vcc. When button is pressed, connection from microcontroller input on which button is connected goes to GND, and button press is registered, but only during the time the column is active. When column isn’t active anymore, it’s connected to Vcc. Since those switches have two LEDs inside, one of which has anode on – pin, when button is pressed, and column isn’t active, that anode is actually connected to Vcc, and LED row on microcontroller starts acting like current sink, so LED is lighted up.

switch

In order to solve this, I’ve placed Schottky diode (to minimize voltage drop) between LED anodes and microcontroller LED row pin, so that the current is blocked for second LED inside switches.

switch2

Demo

So, to conclude this post, here’s a short video of Ceylon in action, together with Tannin.

And also some higher quality pictures of Ceylon.

IMG_8551 IMG_8553 IMG_8565

Thanks for reading!

Assumptions

In this post, I’m going to talk about collection of libraries that make up the OpenDeck platform, and why it’s important to really understand your code well.

Arduino

Before I start talking about Ownduino, I want to clarify what Arduino really is. Basically, it is an Atmel AVR microcontroller placed on a nice PCB with pin headers, combined with libraries that make the programming those chips so much easier, hiding away all the scary parts of embedded programming from user. Once you start dwelling into source code of most of the functions, you’ll find out that most of them have dozens of lines of code for performing really simple tasks, like reading the pins. But that is understandable, it’s the price you pay for convenience. After all, Arduino was never designed to give you the most of its hardware, it was designed to get you into the world of embedded programming really fast. What is not understandable is that most of Arduino code is really bad. Much of the code even goes against recommended approaches in Atmel ATmega datasheet, just take a look at serial functions. There is also a case of enabling many things by default, including lots of unnecessary code you don’t need most of the time. ‘You can’t manage something you can’t control’ is a nice thing I learned this year in college, and it applies perfectly here. Since I wanted to be in complete control over the code that runs on my platform, I created something called Ownduino. Not that creative name, but whatever.

Ownduino

Ownduino is a lightweight library which contains only few of most used functions from Arduino, like Serial.begin(), Serial.write(), millis() and a couple more functions. You can check out rest of the features on Ownduino GitHub. Ownduino also gives user a choice to completely disable ADC, timer or Serial buffer. digitalRead, digitalWrite or String class are examples of what you will not find in Ownduino.

MIDI library

For OpenDeck project I’m using modified Arduino MIDI library v3.2. I realize there is newer version available, but this one has worked flawlessly so far, and I don’t see that changing any time soon. However, I did modify it to more suit my needs. That’s the beauty of open source. Of course, all of the changes I made are published under OpenDeck GitHub repository. Most of the changes were pretty minor, like removing stuff I don’t need, and adding a option to enable or disable MIDI running status via MIDI System Exclusive.

MIDI running status issues

I do very little testing with Altmustech AU-123 MIDI chip, since I’m only using it in finished products (controllers I sell). Because of that, I recently run into some issues when enabling MIDI running status by default. Running status is great in theory, you can read more about it here. However, most MIDI equipment still has issues with it, and it’s best to leave it disabled if unsure what to do. Anyways, MIDI chip I’m using started to misbehave when I enabled it, and it really took me a while before I figured what was wrong, because the same code worked without issues when using serial to MIDI conversions. Reason why this didn’t work is because USB MIDI actually prohibits running status – I didn’t know about that until this had happened. Lesson: never assume things will work. If they do, then there is most likely something wrong. It’s the only constant in universe.

System Exclusive

Now this is a major issue, also discovered recently. It turned out that MIDI chip I’m using doesn’t really support MIDI SysEx, which means that with current OpenDeck setup, you are unable to configure anything using AU-123 – the core platform feature! Chip actually pretends it supports SysEx; it will gladly forward SysEx start and first byte after it, but it will also gladly ignore anything after first data byte, which renders its SysEx ‘support’ unusable. Not all is lost, however. I can still configure controller in emulated environment (virtual MIDI cable + HairlessMIDI + MIDI-OX), and because of this, I’m moving away from current ATmega328P setup to Arduino Pro Micro, with Atmega32u4 on it, which has USB support, so I can get rid of separate MIDI board.

Code optimizations

Recently, I’ve discovered that I can send real time MIDI data from Traktor Pro to a MIDI controller. MIDI data in question is track volume. You can send volume data to controller to turn the LEDs on and off, resulting in VU-meter. The problem with previous version of OpenDeck was that code was too slow. Code used too much time to read the potentiometers, which affected timing of column switching inside matrix and reading of MIDI input. Result of all that was that code was unable to catch all incoming MIDI data, so some LEDs would’ve stayed on when they don’t need to be, and vice versa.

Time interrupts

Thing I learned from this is that matrix switching should always be done inside a interrupt routine. It’s the only way of ensuring correct switching without any delays. Placing matrix switching inside interrupt routine was first thing I did to optimize the code. Second thing again boils down to knowing your code, something Arduino hides away from you.

ADC configuration

In order to explain what I did, it’s necessary to understand how ADC inside ATmega works. When converting analog signal to digital one, ATmega takes samples of input signal at a certain speed, derived from microcontroller clock sped (16MHz). In order to control this speed, there are couple of prescalers available (ATmega328p datasheet, page 256):

ScreenShot086

By default, Arduino uses prescaler 128, which means it runs at 125kHz. Since single ADC conversion takes 13 cycles, that gives us a sample rate of 9.6kHz. Note however, that pushing the ADC clock up to 1MHz doesn’t have much of an influence on 8-bit values, and since MIDI requires 7-bit resolution, there is no reason not to exactly that, that is, to set the prescaler to 32, giving us a sample rate of 38kHz, much faster than the default. Because of this, I included a setADCprescaler function inside Ownduino, to make it easier for me:

 

void setADCprescaler(uint8_t prescaler) {

//disable ADC before setting new prescaler
ADCSRA &= (0<<ADEN);

switch(prescaler)   {

case 16:
ADCSRA |= (1<<ADPS2)|(1<<ADEN);
break;

case 32:
ADCSRA |= (1<<ADPS2)|(1<<ADPS0)|(1<<ADEN);
break;

case 64:
ADCSRA |= (1<<ADPS2)|(1<<ADPS1)|(1<<ADEN);
break;

case 128:
default:
ADCSRA |= (1<<ADPS2)|(1<<ADPS1)|(1<<ADPS0)|(1<<ADEN);
break;

}

}

Now, we get much faster analog read-out, and 8-bit precision. If you consider that ATmega328P has 10-bit ADC, and that ATmega328P is 8-bit microcontroller, ADC value is obviously stored inside two registers: ADCH (2 higher bits) and ADCL (8 lower bits). We can easily discard two bits, so that we need only one register. To do this, we can invert places where ADC value is stored by using this line:

ADMUX |= (1<<ADLAR);

Now, analog value is in the range 0-255, and when reading the value, we simply read ADCH register, eliminating the need to read extra register. Divide it by two, and we get nice 7-bit value required for MIDI.

Conclusion

Together with doing matrix switching inside timer interrupt, code now performs much faster, and there are no more problems with sending lots of data to controller. You can check out VU-meter in action here:

So, in short:

  1. Never assume anything
  2. Know your code

That would be all.

OpenDeck SysEx protocol – stable!

Last time I wrote about OpenDeck SysEx protocol, there were still many unsolved problems and issues, together with lack of functionality. Since I actually released OpenDeck software on GitHub couple of days ago, it’s time to write a new post, explaining all current features, and announcing some new ones.

Hello World

Back when I first wrote about SysEx protocol, I explained how SysEx programming doesn’t work until “Hello World” message has been received. That bit wasn’t implemented then, but it is now. Hello message consists of three ID bytes only (hex code):

F0 00 53 43 F7

After the message is sent to controller, it responds with those same ID bytes, and ACK at the end (0x41).

F0 00 53 43 41 F7

When this message is received, controller can be programmed using System Exclusive.

Message format

Main message format for SysEx is left untouched, as it proved efficient and reasonably easy to use:

F0 ID ID ID WISH AMOUNT MESSAGE_TYPE MESSAGE_SUBTYPE PARAMETER_ID* NEW_PARAMETER_ID* F7

Bytes marked with asterisk are optional, since their use depends on WISH and AMOUNT bytes.

Message types

Since last post, only thing changed here are MESSAGE_TYPE codes:

  • SYS_EX_MT_MIDI_CHANNEL
  • SYS_EX_MT_HW_PARAMETER
  • SYS_EX_MT_FREE_PINS,
  • SYS_EX_MT_SW_FEATURE
  • SYS_EX_MT_HW_FEATURE
  • SYS_EX_MT_BUTTON
  • SYS_EX_MT_POT
  • SYS_EX_MT_ENC
  • SYS_EX_MT_LED
  • SYS_EX_MT_ALL

MIDI channel has code 0, HW parameter 1, and every next byte has next incremented value.

Additions

Pots now have additional sub-types: lower and upper CC limit. Using those message, each pot can have defined CC range. After you define custom range, ie. 40-120, CC message won’t be “cut-off”, it will be interpolated across whole pot range instead.

Another addition from last time is the “Free pins” message, that is, you can now configure those four free pins available on reference PCB. Due to the mistake in design, pin C can only be configured as output, since it’s connected to pin 13 on Arduino. It has nothing to do with pin 13 itself, it’s just the fact that pretty much every Arduino out there has LED and resistor connected to that pin, so using it as input is harder, but not impossible. It would require usage of some extra components, but for convenience sake I’ve decided that it can only be configured as output. Because of that, you cannot increase number of inputs to 64 (four extra rows in button matrix), but 56 instead. No big deal, but I’ll most certainly pick another pin for user configuration on next PCB design.

Example of free pin configuration:

Let’s set pin A as output, which will result in increase of maximum number of LEDs to 40:

F0 00 53 43 01 00 02 00 00 02 F7

F0 is usual SysEx message start, followed by three ID bytes. We want to configure (set) pin, so WISH byte is 01. Since we only want to set one pin, amount it 0 (single). Message type for free pin message is 02, with no sub-type. Pin A parameter code is 00, and since we want to set it as output, new parameter code is 02. Setting it to 0 disables it, and setting it to 1 would configure that pin as extra button row.

Response from the controller is the following message:

F0 00 53 43 01 00 02 00 41 F7

Response has been also re-modeled a bit since last time. It copies everything from the original message, except the parameter and new parameter (if message wish is set, that is). It also places ACK (0x41) on the end of message, so that it’s easier to notice if the message has been handled properly.

Data restoration

Data restoration is now possible as well. You can use it to restore single parameter, all parameters within message type or everything back to default.

Some examples of restoration:

Setting button channel back to default (1):
F0 00 53 43 02 00 00 00 00 F7

Setting all channels to default:
F0 00 53 43 02 01 00 00 F7

Setting all pots to their default state (disabled):
F0 00 53 43 02 01 06 00 F7

Complete factory reset:
F0 00 53 43 02 01 09 00 F7

If entire controller is restored to defaults, first thing that needs to be configured is board type. Board type for OpenDeck reference board is 01, and you need to set it using the following message:

F0 00 53 43 01 00 01 00 00 01 F7

LED configuration

LEDs configuration is another thing that still wasn’t possible when I wrote first SysEx post. LEDs now have several sub-types of message:

  • SYS_EX_MST_LED_ACT_NOTE
  • SYS_EX_MST_LED_START_UP_NUMBER
  • SYS_EX_MST_LED_STATE

Again, LED_ACT_NOTE has code 0, START_UP_NUMBER is 1, and LED_STATE is 2. Using the first subtype it’s possible to select which MIDI note will turn on or off the LED. Start-up number is used only during the start-up routine. Since that routine deals with LEDs one-by-one, you can use that message sub-type to use any LED order you want. LED_STATE subtype is used to test the LEDs. You can set them to off state, constantly on, blinking and blinking off state.

Start-up routine

Start-up routine won’t work until total number of LEDs has been specified. To set it, use the following message:

F0 00 53 43 01 00 01 00 03 TOTAL_NUMBER_OF_LEDS F7

There are currently four routines user can select for start-up:

F0 00 53 43 01 00 01 00 05 START_UP_PATTERN F7

Pattern can be anything 0-4. Setting pattern to 0 will disable start-up routine.

Changing order of LEDs during start-up routine:

F0 00 53 43 01 00 08 01 LED_NUMBER LED_START_UP_NUMBER F7

More examples

There are many more examples listed on OpenDeck GitHub repository, so feel free to check them out.

Planned features

At the moment, I am adding support for encoders in OpenDeck software. Due to their unreliability, it’s really tricky to deal with them in software and getting stable values, but I’m working hard on it, and code should be finished soon.

Demonstration

I recorded a video to show-off all the mentioned features:

Sensimidia MIDI controller

Sensimidia is third Shantea Controls controller, and also my second commercial project. I’ve been working on it for the last two months, but most of that time I did nothing, because of some delivery issues with faders and box. Now that it’s finally over, I’d like to present it here.

SensimidiaIn many ways, Sensimidia is similiar to Anandamidi controller. Same buttons, same pots, same faders, same mounting techniques – except for the faders, which I’ve done right this time. It features 24 buttons, 12 LEDs, 6 rotary potentiometers and 2 faders. I’ve also added two additional mounting screws to the plate to avoid its bending, although I should’ve added two more – one in the middle left, and one in the middle right. Design features logo from the guy who ordered the controller. It turned very cool, but then again, that board has magical capability of turning everything into something super-nice.

This controller is first controller I’ve built using my recently-announced OpenDeck platform, therefore, if you’re interested what hardware does this run, scroll few posts below. Worth of note is that even though it’s based on OpenDeck design, it doesn’t mean Sensimidia runs full version of OpenDeck software. Yes, buttons, pots, MIDI input – all of that works, but full controller configuration via MIDI System Exclusive messages doesn’t work yet. Luckily, that is nothing to worry about in this case, since the guy really wanted something simple that works out-of-the-box. I can always update the firmware on controller later – once the OpenDeck platform is finished, and also assuming the guy would actually want the extra features.

Anyways, here’s how Sensimidia looks all wired-up:

OLYMPUS DIGITAL CAMERAAs I said, it’s all very similiar to Anandamidi construction. This OpenDeck board connects to my MIDI USB PCB via pin header right above that electrolytic cap.

OLYMPUS DIGITAL CAMERAThis is how it looks in case. Very nice, but a bit too dark for my taste.

P1010005_crAnd standard USB plate for the cable.

All in all, very fun project again, even though it’s not been really that much different from Anandamidi. Here’s a quick video I’ve taken demonstrating some of controller functionality (really sorry for the lack of good camera and tripod):

Building OpenDeck – SysEx protocol

MIDI is a great protocol. One if its best features is System Exclusive message – the only type of message in MIDI protocol which doesn’t have defined message length, that is, it can be dozens of bytes long, or more, given that data bytes have MSB bit set to zero (byte value is 0-127). Only thing that is defined is its start, “0xF0” and end, “0xF7”. Manufacturer ID byte (or three bytes) is what usually follows after start byte, so that only specific devices accept the message and respond to it. Everything else is optional, which means you can create your own protocol for configuring your MIDI device using nothing more than a SysEx message. Because of its undefined nature, you can even use it to add more features to MIDI itself, if you find MIDI protocol too limiting for your needs, which is kind of hard to imagine, but oh well.

SysEx is also the basis of configuring OpenDeck platform. I’ve tried to make protocol as simple as possible, so that it’s easy to understand and remember. This is the main message format:

F0 ID ID ID WISH SINGLE/ALL TYPE SUB-TYPE PARAMETER_ID NEW_PARAMETER_ID F7

Whoosh! What’s all that? I’ll try to explain every byte. As you can see, there are three ID bytes after the SysEx message start. Back when MIDI was new standard, specification said only the first byte after SysEx start is ID byte, which due to the SysEx limitation of maximum value of data byte being 127 meant that there could only be 127 MIDI manufacturers in the world (ID 0 is unassigned). That was true in first couple of years, but after a while, single byte just wasn’t enough anymore, so it was decided that three bytes could be used as ID as well. Those three bytes are actually two, since first byte is always zero. Even two bytes (just one byte more than what original specification said) are enough for more than 16k combinations, so there shouldn’t be any reason ever to increase those three bytes to four, or more. Furthermore, there is an ID reserved for educational and development purposes, which is what I should use, or at least what I thought I’d use. However, that ID prohibits use in any product released to the public, commercial or not, so that was not a deal. On the other hand, to use any other ID, you have to register and pay anual price od 200$ to MIDI Manufacturers Association. So, I decided to just use 3-byte ID and hope nobody will complain about it. I’ve checked official manufacturer ID list and luckily, nobody is using the ID I wanted. Also, given how my SysEx system works, it’s basically impossible to use it on any other device, even if it has same ID. I settled for 0x00, 0x53, 0x43 ID. That ID, as well as some other parts of my SysEx protocol is based on ASCII table. 53 in hexadecimal system means “S” in ASCII table, and 43 is “C”, SC for Shantea Controls. Easy.

WISH

Wish can have three meanings:

1) GET: Request data from controller. Code for this is 0.

2) SET: Store new data in controller. Code 0x01.

3) RESTORE: Restores default configuration for specified data, or restores every setting to factory default. Defaults can be checked in code here. Code is 0x02.

 

SINGLE/ALL

This byte specifies whether we want to get/set/restore single parameter, or all parameters for specified message type. Single code is 0, and all is 0x01. Again, really easy.

 

MESSAGE TYPE

There are several message types, and their codes are all based on ASCII table, just like ID:

1) MIDI Channel: 0x4D.

There are 5 channels in total (Button press, long button press, CC channel for pots, CC for encoders and input channel). These are their parameter codes (copied directly from here):

#define SYS_EX_MC_BUTTON_NOTE 0x00
#define SYS_EX_MC_LONG_PRESS_BUTTON_NOTE 0x01
#define SYS_EX_MC_POT_CC 0x02
#define SYS_EX_MC_ENC_CC 0x03
#define SYS_EX_MC_INPUT 0x04

2) Hardware parameter: 0x54

3 hardware parameters in total. Codes:

#define SYS_EX_HW_P_LONG_PRESS_TIME 0x00
#define SYS_EX_HW_P_BLINK_TIME 0x01
#define SYS_EX_HW_P_START_UP_SWITCH_TIME 0x02

3) Software feature: 0x53

7 features. Codes:

#define SYS_EX_SW_F_RUNNING_STATUS 0x00
#define SYS_EX_SW_F_STANDARD_NOTE_OFF 0x01
#define SYS_EX_SW_F_ENC_NOTES 0x02
#define SYS_EX_SW_F_POT_NOTES 0x03
#define SYS_EX_SW_F_LONG_PRESS 0x04
#define SYS_EX_SW_F_LED_BLINK 0x05
#define SYS_EX_SW_F_START_UP_ROUTINE 0x06

4) Hardware feature: 0x48

4 main hardware features. Codes:

#define SYS_EX_HW_F_BUTTONS 0x00
#define SYS_EX_HW_F_POTS 0x01
#define SYS_EX_HW_F_ENC 0x02
#define SYS_EX_HW_F_LEDS 0x03

5) Buttons: 0x42

6) Potentiometers: 0x50

7) Encoders: 0x45

8) LEDs: 0x4C

9) Everything: 0x0A

 

MESSAGE SUB-TYPE

Several types of message require more specific info abut their type: buttons, pots, LEDs and encoders. Subtypes are following:

Buttons: Type (code 0) / Note (code 0x01). Type is for getting/setting info about type of button. Type can either be momentary (code 0), meaning that button will send note off right after it’s released, or it can be “toggle” type, which means that pressing button will send note on, releasing it will do nothing, and pressing it again will send note off. Note is MIDI note number which button sends.

Potentiometers: Enabled (code 0) / Inverted (code 0x01) / CC # (code 0x02). All of these are pretty self-explanatory. Setting enabled bit to 1 enables the potentiometer, and 0 disables it. When poteniometer is disabled, it will not send any data. Setting inverted bit to 1 inverts the CC data coming from the pot, which means that usual 0-127 CC range transforms to 127-0. Useful when you reversed the power/ground lines on potentiometer. CC code can be anything from 0-127.

Encoders: Same logic as potentiometers.

LEDs: For now, LEDs don’t have any sub-type, which is probably going to be changed as I develop the protocol further.

Every other message type doesn’t have sub-type, which means that they require setting of sub-type byte to 0.

 

Parameter ID

ID of component we want to manipulate. Allowed IDs:

1) MIDI channels: 0-4

2) Hardware parameters: 0-2

3) Software features: 0-6

4) Hardware features: 0-3

5) Buttons: 0-63

6) Potentiometers: 0-63

7) Encoders: 0-31

8) LEDs: 0-63

 

New parameter ID

Only needed when setting new value of specified parameter. Allowed IDs:

MIDI channels: 1-16

Hardware parameters: 4-15 for long-press time (400-1500 miliseconds), 1-15 for LED blink time (100-1500 miliseconds), 1-150 for start-up LED switch time (10-1500 miliseconds).

Software features: 0-1 (enabled/disabled)

Hardware features: 0-1 (enabled/disabled)

Buttons: 0-127

Potentiometers: 0-127

Encoders: 0-127

LEDs: 0-127

Examples

This is the scenario: we want to find out on which MIDI channel are buttons sending Note data for regular presses (note that all example bytes are in fact in hexadecimal system, not decimal):

F0 00 53 43 00 00 4D 00 00 F7

Message starts with start code. After that, there are 3 bytes reserved for ID. Next comes WISH byte. Since we want to get data, code is 00. We only want single parameter, so SINGLE/ALL byte is also 00. Message type for MIDI channel is 4D, so that is next byte. MIDI channels don’t have sub-type code, so we set next byte to 0. Finally, since we want button channel for regular press, we set the last byte to 0.

Response is handled in very similiar way as request. After sending that message to controller, result is following:

F0 00 53 43 41 4D 00 01 F7

Response also starts with three ID bytes. After that comes 41 byte. 41 in hex system is “A”, and it’s chosen since it represents ACK signal, meaning that received message is correct request, and that there haven’t been any errors during data retrieval. After ACK signal, response sends message type and subtype, that is, 4D and 00 in this case. Final byte is wanted data. Since default MIDI channel is 1, byte is also 0x01. This is the result in a more graphical way (note that Hairless MIDI doesn’t display F0 and F7 bytes):

Screen Shot 08-30-14 at 09.15 PM

Next example: We want to get all MIDI channels at once.

F0 00 53 43 00 01 4D 00 F7

Very similiar to previous example, except that now we’re setting single/all (6th byte) to 1, because we want all parameters. Note that in this message, parameter ID isn’t needed.

Screen Shot 08-30-14 at 09.19 PM

Result is following:

F0 00 53 43 41 4d 00 01 02 01 02 01 

Again, similiar result as previous message, except that now there are 5 values after subtype signal, because there are 5 MIDI channels.

Now, we want to set CC channel for potentiometers to channel 2 (default is 1).

F0 00 53 43 01 00 4D 02 02 F7

5th byte is now 1, because we are setting a value. 6th byte is 0, because we are setting only one value. Following bytes are message type and sub-type, parameter of channel we want to change (code for CC channel for pots is 02), and a new value for channel, 02.

Screen Shot 08-30-14 at 09.26 PMFirst I moved potentiomer 6 for a while. App shows that CC is sent on channel 1. After that, SysEx message is received which alters CC channel to 2. Response for setting is:

F0 00 53 43 41 4d 00 01 F7 

Again, ID followed by ACK, followed by type/sub-type and final byte 1, which means controller has successfully written one new value. Note that these are permanent changes, that is, all values are written to microcontroller EEPROM, meaning that they won’t be replaced by defaults next time controller starts-up.

There is another type of message, “hello world”, used for setting up SysEx communication with controller:

F0 00 53 43 F7

If we send this message to controller, the response we would get is:

F0 00 53 43 41 F7

Which is ID followed by ACK signal. This is used for checking the device ID. If this check fails, no further SysEx communication will be allowed until the “hello world” message has correct ID.

Error messages

Each SysEx message goes through rigorous testing of each supplied byte, so it’s impossible to mess up controller with wrong values. Every time message format is wrong, controller will send error message with error code. For now, these are error codes:

Error 0: F0 46 00 F7

This is error code for wrong device ID. When you supply wrong ID code, first byte after the start is 46, which is “F” in ASCII table (F for failure), which means something has gone wrong. After error byte, next byte is error number, in this case 0. This error is the only one in which device ID isn’t sent, for obvious reasons.

Error 1: F0 00 53 43 46 01 F7

Error code for wrong WISH command. Note ID in response.

Error 2: F0 00 53 43 46 02 F7

Wrong single/all command.

Error 3: F0 00 53 43 46 03 F7

Wrong message type.

Error 4: F0 00 53 43 46 04 F7

Wrong message sub-type.

Error 5: F0 00 53 43 46 05 F7

Wrong parameter ID.

Error 6: F0 00 53 43 46 06 F7

Wrong new parameter ID.

Error 7: F0 00 53 43 46 07 F7

Message too short.

Error 8: F0 00 53 43 46 08 F7

Error while writing new data to microcontroller.

Keeping data integrity

OpenDeck code makes sure that new data is actually written to EEPROM without issues. Here is example of checking for whether MIDI channel has been written successfully:

bool OpenDeck::sysExSetMIDIchannel(uint8_t channel, uint8_t channelNumber) {

switch (channel) {

case SYS_EX_MC_BUTTON_NOTE:
_buttonNoteChannel = channelNumber;
eeprom_update_byte((uint8_t*)EEPROM_MC_BUTTON_NOTE, channelNumber);
return (channelNumber == eeprom_read_byte((uint8_t*)EEPROM_MC_BUTTON_NOTE));
break;

...

...

}

}

Function first overwrites channel currently in RAM. After that, it writes new value to EEPROM. Finally, it checks if received channel is equal to the one just written in EEPROM, so that it’s impossible to get ACK message if this step fails.

That would be all for now. Protocol is still in heavy development, and some of the stuff I’ve talked about here isn’t even implemented yet.

Building OpenDeck platform, part 1

I’ve been real quiet about this project for more than a month, but not because I haven’t done anything. Quite the opposite, I literally got consumed by work invested in this project. I realized there’s much more stuff to do than I thought. As I mentioned last time, the project consists of three distinct parts:

  1. PCB
  2. MCU software
  3. GUI application

Apart from some mockups, there’s nothing else I’ve done with GUI for the time being, so I’ll just talk about first two points.

1) PCB

I’ll repeat what I said about PCB last time: it has 32 digital inputs, 32 digital outputs and 16 analogue inputs. As far as digital stuff is concerned, it’s a simple 8×4 matrix with shared columns for both buttons and LEDs. The column switching is done via 74HC238 decoder. Its outputs are connected to ULN2803 transistor array, since Arduino alone can’t handle that much current sink when all 4 LEDs are turned on in a column. Button and LED rows are connected directly to Arduino (with 220Ohm resistor for each LED row). 16 analogue inputs are handled with two 4051 multiplexeres. All in all, fairly simple circuit. Bonus features I’ve came up with are 4 additional configurable I/O pins, which you can configure in GUI app (but not right now, since GUI isn’t finished yet, and there’s not even support for those pins in MCU code). So in essence, you could use those four pins to add two additional rows in button matrix and two rows in LED matrix, resulting in total of 48 buttons/LEDs. Or you can use all four to add 32 additional buttons. Or LEDs. There is some flexibility involved in any case. Here’s how the PCB looks:

OLYMPUS DIGITAL CAMERA

Since I’m already using it in a project I’ll talk about soon, I can tell you that everything works perfectly. No wrong connections or any electrical problem. I did manage to reverse poteniometer labeling though: pots 0-7 are actually 8-15, and vice versa. Annoying? Kinda. Deal breaker? Most certainly not. I’ve also added ground shortcuts to multiplexer inputs, so if you’re not using one of multiplexers (or both), you can quickly shortcut them to GND to avoid floating pins.

OLYMPUS DIGITAL CAMERA

That’s how the board looks like with most of the stuff soldered. There is one (hardly) visible issue: Right above of the ‘A’, ‘B’, ‘C’ and ‘D’ pins (which are those configurable pins I’ve talked about) are GND pins. I intended it to be quick way to shorcut those four pins to GND if they’re left unused, since I don’t like having any pins floating. Now, there is nothing inherently wrong with that idea, except that you can’t simply ground some pins on ATmega chips. I’ve learned it the hard way. After I’ve done all the soldering, naturally I’ve connected all four of those pins to GND since the project I’m using this board for at the moment doesn’t require them. I’ve connected the board to USB via CP2102 serial-to-usb converter, but the LED connected to pin 13 on Arduino didn’t blink at all. After you connect the board to USB, that LED blinks for 6 seconds, after which your code starts running. In this case, only the power LED got turned on, so there was apparently something fishy going on. First I simply tried to upload code to the board using serial converter, but that didn’t work. I think the message in avrdude said something along the lines of “board is not responding”, or similiar. Weird. After that, I assumed that maybe I accidentally removed the bootloader while I was playing with the Arduino couple of days earlier. I connected my USBASP programmer to the board (or more precisely, soldered the cables to pins sticking out since I had no other way of doing it) and decided to simply burn the bootloader onto the board, but that also didn’t work. I got similiar message like when I tried to upload new code via serial converter. Damn. After that, I started to wonder that maybe 10nF electrolytic cap and 0.1uF cap are too much capacitance, causing the board not to work properly, because I had the same scenario with my USB MIDI PCB (you can read about it couple of posts below). So I simply cut the 0.1uF cap at first to see what happens (that’s why the cap right next to the electrolytic cap may seem a bit weird, because later I’ve re-soldered it). Nothing. Then I almost cut the electrolytic cap as well, when it hit me. When you upload new code to your board via external programmer, you connect MOSI, MISO, SCK, reset, VCC and GND pins from programmer to ATmega. MOSI, SCK and MISO are pins 11, 12 and 13 on Arduino board. The unused ‘A’ and ‘B’ pins are actually pins 12 and 13 on Arduino. See the problem here? How the hell am I supposed to upload new code to board via pins which are connected to GND? Then I simply removed the solder shortcuted to GND pins, and magically, board worked. Another lesson learned. Apart from that, there were no other issues.

2) MCU code

There’s been many changes and optimizations in code since the last time I posted. I removed HardwareReadSpecific library since hardware control is being handled via callbacks now, so all the configuration is being done within main program now. There is added benefit of being able to compile the whole OpenDeck library as a single file and simply including it as precompiled binary in other projects, but since it’s still under heavy development, I wouldn’t have any use from it. I’ve also made a “database” of parameters in EEPROM, like MIDI channels, blink time, button notes, CC numbers etc. There’s also array of default parameters in code, so that there’s an option of performing “factory reset” of MIDI controller. I’ve saved those default parameters in PROGMEM part of ATmega memory, since I would’ve wasted limited amount of available SRAM otherwise. 1kB of EEPROM may sound like tiny amount of space, but in reality it really isn’t. You are free to check the code on my GitHub page. I still have to design my own SysEx protocol to program the controller. I hope I’ll manage to finish that in the next few days, after which I can move on to GUI app.

That would be all for now.