A Simplified Constrained Bayesian Optimization Approach to Optimize the Tx Equalization in SerDes Channels

Design of high-speed channels has become increasingly more complicated. Due to the eye diagram closure at higher speeds, designers use Tx equalization by placing a finite impulse response (FIR) filter at Tx. Assigning the FIR tap values can be time consuming and require domain expertise since it can require sweeping hundreds or more combinations of tap values. Therefore, in this letter, we propose a machine learning optimization approach to find the FIR tap values which result in the largest eye opening. Conventional optimization techniques may not be applicable in this context since specifications of the channel can require a constraint, which is the sum of the absolute value of the FIR taps needs to be equal to 1. Therefore, we have developed a simplified constrained Bayesian optimization (BO) approach that can automate this process and expedite calculation of the FIR tap values without requiring domain expertise. Numerical examples are provided to show efficiency of the proposed approach and compare its performance with BO and genetic algorithm for this problem.