moteus-n1 gate drive strength fix

Sometimes a bug report comes in that is simple. Sometimes a bug report comes in that is complex. This is the story of a bug report that looked very simple, but in the end was actually much more complex than I had initially expected. Further, it likely contributed to electrical damage on many deployed moteus boards, which is definitely not something I aim to do! Read on to learn more.

The public version of the story starts with the filing of github issue 114. In that issue, the reporter noticed that the firmware code for moteus was using the incorrect gate drive strength mapping table for boards with the DRV8353 gate driver.

I had actually discovered the same issue privately during an automated audit run at approximately the same time. I had Claude use a technique similar to one I had seen published for Firefox auditing. I had it run in a batch fashion, over each file in the moteus repository individually, with a prompt to the effect of “look at this file line by line looking for correctness or security issues”. That resulted in maybe a hundred different fixes all of which were relatively easy to apply except for this one.

Context

For some context, moteus boards use MOSFETs to switch power to each of the phase wires that ultimately drive a motor. The electrical signal required to drive an appropriate power MOSFET is more than a typical microcontroller can supply, so moteus boards use an integrated gate driver to perform that function along with some other functions that most motor controllers need. Different moteus boards have used two different variants of gate driver, either the DRV8323 or the DRV8353. The DRV8323 is best suited to slightly lower voltage and slightly smaller MOSFETs. It is used on the moteus-c1 and moteus-r4.5 and older boards. All other moteus controllers use the DRV8353.

One of the functions the DRV83X3 gate driver family supports is configuring how much current to use to charge and discharge the gate terminal of each power MOSFET. Selecting an appropriate value is a balance between reducing switching losses, and minimizing electromagnetic interference and risk of electrical damage. Lower values of current make the MOSFET switch on or off slower, which means that they spend more time in the “linear region” where they are busy converting lots of power into heat. However, this slower turn on and off also decreases the slew rate of the switching event which reduces the magnitude of ringing and overshoot at the gate and phase terminals. This decreases electromagnetic interference generated by moteus and reduces the likelihood that ringing at a phase terminal or gate terminal will exceed the absolute maximum voltage of the MOSFET or gate driver.

Implications

The implications for this particular issue were that moteus was using the wrong table to convert milliamps of configured drive current into the small numbered integers used to program the gate driver. Thus, the gate driver was not configured for the drive strength that was intended. This could be bad, but in practice, it didn’t cause too many issues because the selected values were derived from empirical tuning using the incorrect gate drive mapping. Thus, after fixing the table, moteus_tool could also fix the configured values so that the gate driver applied the same gate drive strength as before and newer firmware just defaults to those same new strengths.

However, looking at this bug discovered a different, more severe issue. Recall that the moteus-c1 uses a DRV8323, which uses entirely different gate drive strenth mappings from the DRV8353. When it was introduced, the firmware was intended to be updated to use the DRV8323 mappings when running on a moteus-c1. However, the code to select which table to use was incorrect and actually switched the moteus-n1 to use the DRV8323 tables and left the moteus-c1 using the DRV8353 tables. The moteus-n1 controller had its tuning done with the DRV8353 tables, so the net effect of switching it to the DRV8323 tables was that the MOSFETs were driven with a higher current than was intended by the design. This had a slight benefit of reducing thermal loss, but a much bigger downside that the controller was much more susceptible to electrical damage caused by improper phase wire soldering and emitted more EMI than intended.

The Fix

The fix for the firmware side is relatively straightforward. When upgrading to firmware version 1.1.x or newer, moteus_tool will update the configured gate drive strengths to be the value that maintains the same strength for all non-moteus-n1 controllers. For moteus-n1 controllers, it will update the gate drive strength to be the initial validated value. To maintain similar thermal performance at the expense of peak power, it will modify the moteus-n1 PWM rate to 20kHz. Thermal performance is the most common performance limiter rather than peak power, which is why this route was chosen here.

Further, the moteus performance analysis tool, or mpat, was developed after this bug was introduced. Thus the thermal performance profile for the moteus-n1 was collected using the incorrect gate drive strengths. I went through the process of collecting a new performance profile, and as of this commit, the reported performance and default PWM rate matches that of firmware 1.1 and newer.

As to damage to existing controllers, problems are largely restricted to cases where the supply voltage was 36V or higher and the phase wire terminal soldering was “not ideal”. Despite that, if you have purchased a moteus-n1 in the last year, you should have already received an email inquiring about refunds or replacements for any units that have experienced gate driver or MOSFET damage.

Lessons for the future?

I guess this is another issue in the series “hardware is hard”? The problem ultimately came with the release of the moteus-c1, and it isn’t clear, aside from a better job of detailed source or patch inspection, that any reasonable set of regular test procedures should have been in place that would have caught the regression in robustness of the moteus-n1. Performing gate drive strength characterization is a relatively involved part of the release process for a new board and would be prohibitive if it had to be re-performed across all existing boards. moteus is open source, but like most open source projects, receives almost no outside human eyeballs looking for bugs.

The best thing I have going is that an automated AI scan did identify the problem, so perhaps the best takeaway is that a similar process can be instituted on a semi-regular basis using the current best models available?