Low latency control board for LLRF system – SIMCON 3.1. Wojciech Giergusiewicz1, Wojciech Jałmużna1, Krzysztof Poźniak1, Nikolay Ignashin3, Mariusz Grecki2, Dariusz Makowski2, Tomasz Jeżyński2, Karol Perkuszewski1, Krzysztof Czuba1, Stefan Simrock3, Ryszard Romaniuk1 1 -Warsaw University of Technology, 2-Technical University of Lodz, 3-DESY, Hamburg
[email protected] ABSTRACT A new version of the SIMCON system is presented in this paper. The SIMCON stands for the microwave, resonant, superconductive accelerator cavity simulator and controller (embracing the hardware and software layers). The current version of the SIMCON is 3.1. which is a considerable step forward from the previous 8-channel version 3.0. which was released at the beginning of 2005 and was made operable in April. Many important upgrades were implemented in SIMCON 3.1. It is a stand-alone VME board (whereas SIMCON 3.0 was modular) based on the Virtex II Pro 30 chip with two embedded Power PCs and DSP blocks. It has Ethernet and multiple gigabit optical I/Os. The Simcon 3.1 board provides 10 ADC channels. The architecture idea and block diagrams of the PCB for SIMCON 3.1. are presented. Some of the applied novel technical solutions, Protel® views and schemes are shown. A number of initial conclusions were drawn from a few month experience with the development of this new board. The tables of predicted system parameters are quoted. Keywords: SIMCON system, LLRF control system, superconducting linear accelerator, free electron laser, FEL, VUV FEL, X-Ray FEL, advanced PCB, PCB architecture
1. INTRODUCTION The paper describes a continuation of the efforts of several closely cooperating research groups from Warsaw, Lodz and Hamburg on the next generation of the Low Level RF (LLRF) control systems for the superconducting linear accelerators based on the TESLA technology (narrowband, high finesse, niobium cavities). These accelerators, now under construction and planned for the future, are the propelling engines for the UV Free Electron Laser (FEL) (scheduled soon to be a user facility), X-Ray FEL and the planned immense International Linear Collider (ILC). Now, most of the accelerators are controlled by the standard DSP VLSI chip generation of the LLRF systems. The DSP systems have several important advantages, but with the advent of powerful FPGA chips with DSP capabilities and optical I/O turned out to be lagging behind in the performance, where the speed, control loop latency and beam control precision (via the phase and amplitude stability of the accelerating field) are of the major importance. The international LLRF group, localized in DESY, is aiming at building of a new generation of the LLRF system, capable within several nearest years to replace the old generation one. The replacement is planned for all control system components like: LLRF high performance 1,3 GHz cables and spacious patch-panels will be replaced by multi-gigabit fiber optic network, the penetration of fast digital technologies inside the control system will be considerably increased, the resignation from the power consuming VME standard is considered, particular parts of the system will be replaced from the VME crate clusters (located far away from the accelerator bunker, thus, introducing larger signal transmission latency) to other positions closer to the klystron and the cavities, etc. The design process of the advanced multilayer PCB for the SIMCON 3.1. system embraces the following sections: VME interface, power supplies and static RAM, Virtex II Pro and dynamic RAM, Opto-gigalink interfaces, DAC and analog output section, Clock distribution system, timing signal inputs, Ethernet interface, tracks impedance matching, digital I/Os, EMI and parasitic signal coupling avoidance, optimal chip distribution on the board, optimization of routing, bandwidth confinements, etc. The PCB cost minimization was omitted this time. The SIMCON 3.1. project was started in February 2005 as a very fast, 4 months one. The brave time schedule was justified by the high performance obtained in the SIMCON 3.0.
Photonics Applications in Industry and Research IV, edited by Ryszard S. Romaniuk, Stefan Simrock, Vladimir M. Lutkovski, Proc. of SPIE Vol. 5948, 59482C, (2005) · 0277-786X/05/$15 · doi: 10.1117/12.622481
Proc. of SPIE Vol. 5948 59482C-1 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 04/13/2015 Terms of Use: http://spiedl.org/terms
2. ARCHITECTURE AND BLOCK DIAGRAMS OF SIMCON 3.1. PCB Fig.1. presents a general diagram of the SIMCON 3.1. PCB. The system contains the following blocks:
INPUT
x10
x10
x4
DAC
ADC x3
~
TIMING DISRIBUTION
DIGITAL INPUT FOR TIMING SIGNAL
INT. CLK
x2 DIGITAL OUTPUTS
x2
FPGA
OPTO GIGA-LINK TRANSCEIVERS OPTO ANALOG DIGITAL INPUT
OUTPUT
x1 x2 x1
VME
PROM
SRAM DRAM
ETHERNET Fig. 1. A general architecture of the SIMCON 3.1 PCB
Below there are particular functional blocks of this architecture described. The functional structure of the board bases on a central large FPGA XC2VP30 Virtex-IIPro chip by Xilinx. The board possesses numerable peripheral circuits. The following functionalities were predicted: • 10 input analog broadband channels of 270 MHz and the voltage range ±1 V (matched to the wave impedance of 50 Ω). The input signals were connected to the individual MCX sockets. Each channel is equipped in an input AD81139 amplifier. The amplifier buffers the asymmetric input signal and transforms it to a differential one matched to the requirements of the ADC. Such a solution minimizes the interchannel crosstalk on the PCB. Fast AD6645 circuits by Analog Devices were used as the ADCs [9]. The maximum sampling frequency is 105 MSPS for a 14-bit word. The digital and analog parts in the ADCs are physically separated. The converters have the reference voltage source and the Sample & Hold system. The catalog data give the SNR value as 74 dB. • 4 analog output channels of 68MHz bandwidth for maximum signal ±1 V (load 50 Ω). The 14-bit ADCs (AD9744 by the AD [9]) were applied in the design. The maximum conversion frequency is 210 MSPS. The circuit AD8139 was applied as a desymetrizing output buffer. The output signals are available via the MCX sockets. • There were implemented 4 differential clock inputs in the LVPECL standard. Each input is connected to two MCX sockets and then buffered. • There are made available two auxiliary asymmetric digital inputs and outputs in LVTTL standard. Each signal is connected to a separate MCX socket. • There are two gigabit optical transceivers by Finisar (previously Infineon) # V23848-M305-C56 of the maximum throughput 2.125 Gbps. The short wavelength window is used for transmission 850nm. Data is transmitted through the multimode optical fiber terminated with LX connectors.
Proc. of SPIE Vol. 5948 59482C-2 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 04/13/2015 Terms of Use: http://spiedl.org/terms
• • •
There was applied a specialized optical analog link matched to transmit clock and synchronization signals. These signals are distributed in the whole accelerator control system and the VUV-FEL machine. The output channel uses HFBR2416 photo detector by HP and LT1223 amplifier by Linear Technology. There was implemented a standard Ethernet 100T link using BCM5221KPT interface by BroadCom. The circuit provides service for the physical layer of the Ethernet protocol. The logical layer is realized in the Xilinx VirtexIIPro. There was realized a VME-BUS interface in a separate FPGA circuit ACEX100K by Altera and buffering circuits. The interface is presented in fig.2. The module is initialized automatically from the EEPROM after switching on the power supply. It enables communication with the VME controller, remote configuration of the FPGA Virtex-IIPro (in series mode and fast parallel mode). It serves also as a converter of the communication channel between the Virtex-IIPro circuit and the VME-BUS. JTAG Corniector
JTAG Coirnectoi
JTAG CHAIN
Xilinx Vutex II Pro XC2VP3III
Ext emal
(additional) booting clock 40MHz
Fig. 2. Block diagram of VME section on the SIMCON 3.1. PCB
•
•
There were positioned two external RAM memories on the board: • CY7C1386B – static, of capacity 512K of 36-bit words; The external memory supplements the FPGA resources, • MT48LC4M16A2TG-7E – dynamic, of capacity 128Mb and the word width of 32-bit; It is used by the embedded Power PC processor inside the FPGA circuit. There was realized a complex distribution network for the clock signal. The system bases on the fanout-buffer type ICS8534-01 circuit. The system should provide jitter not bigger than 0.2ps (RMS). A schematic diagram for the clock signal distribution was presented in fig.3. Depending on the needs, the source of the clock signal may be the FPGA circuit co-working with the LO of 50MHz. The LO is provided on the board or the reference signal may be
Proc. of SPIE Vol. 5948 59482C-3 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 04/13/2015 Terms of Use: http://spiedl.org/terms
connected externally. Such a solution provides individual connections to differential clock signals in the LVPECL standard to all ADCs and DACs.
mt eLK
CHSEL
Ext -
Clock
ECL
LVPECL
LVPECL PECL CLK DRIVER
2
OUTP UTS
to ADC's and DAC'S
EXT CLK
LVPECL 2
Fig.3. Clock distribution system on the SIMCON 3.1. board (differential lines are used).
•
•
The Xilinx circuit is designed for DSP applications, control of the peripheral circuits, communication with the computer system and fast optical data transmission. The XC2VP30 circuit possesses around 200 fast multiplication blocks 18x18b. These blocks enable fast hardware based DSP algorithms. Two embedded PowerPC processors with the clock frequency 400MHz enable the computer based DSP calculations, communication servicing, data exchange over the Ethernet protocol. The FPGA circuit features over 30000 programmable logical blocks (4 input logic cells) and 2Mb embedded RAM memory. These resources provide hardware basis for building of complex control layers, monitoring and internal data acquisition. The PLL blocks are used to realize fast pipeline data processing. There are two fast SERDES blocks used to realize optical transmission with the rate of 3.125Gb/s. Such a solution enables flexible networking interconnections between several PCB. There was implemented a complex power supply block. Its structure is presented in fig.4. The original power supply source has only a single voltage of 5V. This voltage is obtained from the VME bus. Thus, it is possible to connect the board to the external power supply, to use the PCB with control over the Ethernet. It is not necessary to place the PCB in the VME crate. Two sections are distinguished in the power supply block. The stabilization in the
Proc. of SPIE Vol. 5948 59482C-4 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 04/13/2015 Terms of Use: http://spiedl.org/terms
analog power supply part is done in a two stage manner. First, the voltage is converted in the DC-DC circuit and then is stabilized in the linear circuits. The digital part of the power supply block contains only DC-DC converters.
+5V From VME
+5V
+5.5V +5.5V
+5V
-6V -6V
DC-DC power converter
AVCC5V/4A
+5V
AVCC-5V/1A
-5V
Linear regulator
+5V AVCC3.3V/200mA
Enable
+3.3V
VCC5V/500mA
+5V
VCC1.8V/20mA
+1.8V +5V
VCC3.3V/2A
+3.3V +5V
+5V only from VME – easy switch to stand-alone mode of operation
VCC2.5V/1.5A
+2.5V +5V
VCC1.5V/4.5A
+1.5V
Fig.4. Power supply section on the SIMCON 3.1. PCB.
4. FUTURE DEVELOPMENT AND CONCLUSIONS The design and manufacturing of the SIMCON PCB is based on the most up to the date components and technologies. The just routed board has the paths of 5 mils, and uses so-called blind vias and buried vias technology. High performance operational amplifiers AD8139 are used. The table 2 compares the parameters of previous version of SIMCON 3.0 [x] with the current one (the SIMCON 3.0 PCB used AD8138 operational amplifier). Table 2. Comparison between the SIMCON 3.0. and 3.1. parameters
ifiput bandwidth
Jiiput Voltage Noise
AIH:
SIMCON3.O ADS13S SIMCON3.1 AD8139
Offset Voltage
U0ff
'out
drift
max
mV
320
5
1
410
225
0.150
4 125
Proc. of SPIE Vol. 5948 59482C-5 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 04/13/2015 Terms of Use: http://spiedl.org/terms
90 100
Dedicated ADC # of bits (up to) 16 18
A considerable number of components and analog circuits positioned on a single VME 6U board forced the design team to use a novel approach to the components distribution. The components of the analog input and output channels were positioned equidistantly around the main FPGA Virtex II Pro circuit. Thus, the shortest paths were provided between these components. The output and input analog MCX sockets are placed around the Virtex II Pro chip – and not at the PCB edge as they may be expected in more classical solutions The signals from these sockets will be transmitted to the front end panel of the VME board via the high quality „microcoaxial” cables. The clock signal distribution circuit was positioned at the opposite side of the PCB exactly in relation to the FPGA. Thus, the clock is also distributed equidistantly to the critical components of the PCB (ADC and DAC). This provides the same time of signal sampling by the converters.
ACKNOWLEGMENT We acknowledge the support of the European Community Research Infrastructure Activity under the FP6 "Structuring the European Research Area" program (CARE, contract number RII3-CT-2003-506395). Authors would like to acknowledge the technical support from the following persons: Krzysztof Kierzkowski from Institute of Experimental Physics, Warsaw University; Frank Eints and Bibiane Wendland from DESY.
REFERENCES 1.
K.T.Pozniak, R.Romaniuk, K.Kierzkowski: “Parameterized Control Layer of FPGA Based Cavity Controller and Simulator for TESLA Test Facility”, TESLA Technical Note, 2003-30 2. K.T.Pozniak, T.Czarski, R.Romaniuk: “Functional Analysis of DSP Blocks in FPGA Chips for Application in TESLA LLRF System”, TESLA Technical Note, 2003-29 3. T.Czarski, K.T.Pozniak, R.Romaniuk, S.Simrock: “TESLA Cavity Modeling and Digital Implementation with FPGA Technology Solution For Control System Purpose”, TESLA Te 4. chnical Note, 2003-28 5. T.Czarski, R.S.Romaniuk, K.T.Pozniak S.Simrock “Cavity Control System Essential Modeling For TESLA Linear Accelerator”, TESLA Technical Note, 2003-08 6. T.Czarski, R.S.Romaniuk, K.T. Pozniak “Cavity Control System, Models Simulations For TESLA Linear Accelerator ”, TESLA Technical Note, 2003-09 7. W. Petersen “The VMEbus Handbook” 8. http://www.xilinx.com/ [Xilinx Homepage] 9. http:// www.nallatech.com/ [Nallatech Homepage] 10. http:// http://www.beyondlogic.org/epp/ [EPP - Enhanced Parallel Port description]
Proc. of SPIE Vol. 5948 59482C-6 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 04/13/2015 Terms of Use: http://spiedl.org/terms
