Fysikalisk
Systemteknik
Fysikalisk Systemteknik
Christian Bohm
Overview:
About the group
Overview of projects
What is an FPGA
Our major projects
Instrumentation seminar- 2003-03-20
1
Fysikalisk
Systemteknik
Fysikalisk Systemteknik
Personell:
Professor Christian Bohm
Lecturer Sam Silverstein
Part time lecturer Magnus Engström
Adjunkt Eddie Ahlestedt
Forskningsingenjör (emeritus) Hans Eriksson
Forskningsstuderande:
Jonas Klereborn
Abdelkader Bousselham
Attila Hidvegi
Florian Bauer (external)
New
Instrumentation Physics
Experimental Physics
Technology
We collaborate with experimental physics groups, focusing on the
development of new instruments.
Make it easier to develop and maintain useful engineering skills
while retaining an active grasp of the relevant physics.
Use experience from projects to solve new problems.
Look for general solutions and methodologies which are easier
to carry over to new problems.
Instrumentation seminar- 2003-03-20
2
Fysikalisk
Systemteknik
Different instrumentation projects:
In collaboration with particle physics, SU
• RD-16 FERMI and RD27 first level trigger
• Digitizing electronics for ATLAS TileCal
• The Jet/Energy-sum processor for the
ATLAS first level trigger
In collaboration with physicists at KI
• Development of a SPECT camera
• Development of a PET-Camera
In collaboration with molecular physics, SU
• Frequency stabilisation of semiconductor
lasers for laser trapping and cooling
of atoms
In collaboration with astroparticle physics, SU
• Participation in the development of IceCube
Instrumentation seminar- 2003-03-20
3
Fysikalisk
Systemteknik
What is a Field Programmable Gate Array?
Configuration
memory
Firmware
Logic with
Data path switches
Instrumentation seminar- 2003-03-20
4
Fysikalisk
Systemteknik
FPGA
Configuration memory
111111111-------1--1---11111111-------1---1-111-11-1
b
c
e
d
f
Programmable interconnect
a
g clk
h
Configurable Logic Blocks
Programmable IO-blocks
Instrumentation seminar- 2003-03-20
5
Fysikalisk
Systemteknik
Configuration memory pattern defines circuit
001011100-------1--0---01000000-------1---0-110-1010
a
b
c
e
d
f
a
g clk
b
h
c
e
d
clk
Instrumentation seminar- 2003-03-20
6
Fysikalisk
Systemteknik
Modifying the memory content changes the circuit
001101100-------0--0---11111000-------0---1-000-1010
a
b
c
e
d
f
g clk
b
a
h
e
d
clk
Instrumentation seminar- 2003-03-20
7
Fysikalisk
Systemteknik
FPGAs have been around since mid-1980s
Early components were programmed at
a bit-level using graphic editors
Increased complexity required better methods:
High level languages (VHDL), or
Schematic specifications
Instrumentation seminar- 2003-03-20
8
Fysikalisk
Systemteknik
When designing complex circuits with FPGAs
one has to consider:
Does the design fit?
Is it fast enough?
Is it too expensive?
Instrumentation seminar- 2003-03-20
9
Fysikalisk
Systemteknik
FPGA design process
High level description (VHDL)
Functional simulation (no timing)
Select FPGA type
Synthesize – translate to simple
primitives (compilation)
Simple timing simulation
Place and route
Full timing simulation
Instrumentation seminar- 2003-03-20
10
Fysikalisk
Systemteknik
State-of-the-art FPGAs
•Very complex (Xilinx, Alterra)
many gates ~ 8 million gates
many i/o pins ~ 800 (400 diff)
flexible interconnects ~ 7 metal layers
high costs ~ 50 kkr
•Multiple clocks - 8
•Embedded memories – 4 MB
•Embedded multipliers - 200
•Embedded processors – 4 PowerPC
•High speed IO – 16 x 3 GB/s
Instrumentation seminar- 2003-03-20
11
Fysikalisk
Systemteknik
Efficient tools required
• Re-use of previously developed code blocks
Intellectual Property blocks
• IP-blocks can be:
In-house developed
Commercially available
Freely available – open-core
• Part of the design can be accomplished by
assembling compatible IP-block
Processors (embedded or IP)
Memories
Busses
Interfaces
Etc.
Instrumentation seminar- 2003-03-20
12
Fysikalisk
Systemteknik
When designing complex FPGA modules
one must decide:
• What to implement in logic
• What to implement in processor software
Hardware software co-design
VHDL  System-C or Handle C
Instrumentation seminar- 2003-03-20
13
LHC physics looks for rare
events – 1 in 1014
High event rates and
High selectivity
Data from entire
detektor but with low
spatial resolution and
reduced dynamic range
from calorimeters and
muon detector
ATLAS data flow
About 100 million channels
new data every 25 ns
40M Hz
Fysikalisk
Systemteknik
Data from ROIs with
high spatial resolution
and full dynamic range
from all subdetector
75 -100 kHz
1 event in 10000
Since all data must be
stored while waiting
for the L1 decision the
L1 processing must be
quick – 1ns
Entire detector with
high spatial resolution
and full dynamic range
from all subdetector
1 kHz
1 event in 100
10 Hz
1 event in 100
Instrumentation seminar- 2003-03-20
14
Fysikalisk
Systemteknik
ATLAS first level trigger
collaboration with particle physics SU
Looks for typical features for event selection
Analog
Input
signals
Muon
Trigger (Italy)
L1 accept
Central
Trigger (CERN)
Calorimeter
trigger
ROI info
The calorimeter trigger is a Birmingham-London
-Rutherford-Stockholm-Heidelberg-Mainz
Looks for isolated clusters resembling single
collaboration
64x64x2
Analog
signals
Preprocessor
(Heielberg)
Electrons/hadrons in the ECAL and HCAL
64x64x2
8-bit
32x32x2
8-bit
Digitizes
determines
amplitudes
and pulse starts
Calorimeter
Electron/Tau
trigger
Processor (GBR)
Jet (Sthlm) and
Missing energy (Mainz)
processor
Looks for energy
clusters
Looks for energy
balance
Instrumentation seminar- 2003-03-20
15
Fysikalisk
Systemteknik
Analog
Calorimeter
LAr, Tile
S
Tower 0.1 x 0.1
Realtime
data path
Pre-Processor
Timing alignment
10-bit FADC
FIFO
BCID
Lookup Table
BC-MUX
Sum 2x2
Cluster Processor
.1 x .1
(e/g and t/had)
Cluster Finding
Count
.2 x .2
Jet/Energy-Sum
Processor
ET EX EY
SET, ET
Jets
Count
Pre-Processor
RODs (DAQ)
Region Of
Interest
Builder (L2)
Level-1
CTP
CP/JEP
RODs
(DAQ)
Instrumentation seminar- 2003-03-20
16
Fysikalisk
Systemteknik
The JET/energy sum trigger
Look for .4x.4, .6x.6 and .8x.8 energy clusters
centered around a local .4x.4 maximum
Form global sums of total Et and missing Et
Process ~1024 .2x.2 jet elements in parallel
All requiring neighborhood information
32 processor boards with large FPGA for Jet
and missing energy processing, sharing
overlapping environment data
Latency (processing time) 200 ns
Many different
module types
Standardized
modules
Instrumentation seminar- 2003-03-20
17
Fysikalisk
Systemteknik
The JET trigger
We have built a 18 layer backplane with
>20 000 pins for the Jet and the E/t processors
VME - Communication
with neighbors
Report results
Instrumentation seminar- 2003-03-20
18
Fysikalisk
Systemteknik
The JET trigger
We have participated in the design of the JEM
Processor board.
And developed firmware for the algorithms
and control functions
Instrumentation seminar- 2003-03-20
19
Fysikalisk
Systemteknik
Experiences from the trigger project:
Large scale system design
Massive pipelined parallel processing
Reliability
Large FPGA design (> 1 Mgates)
Draw on experience from earlier bit-serial
trigger project to do pipelined processing
of multiplexed data -> more efficient use
of logic and interconnects!
Instrumentation seminar- 2003-03-20
20
Fysikalisk
Systemteknik
The ATLAS TileCal Digitizer
collaboration with particle physics SU
Tile Calorimeter
Drawers
Superdrawer
PMTs
Drawer
cross-section
Fiber
HV-control
3-in-1
Module
Light mixer
PMT
3-in-1 mother board
Particle
Digitizer
interface board
Tile_DMU
TTC-rx
Tile_DMU
Tile_DMU
TTC-rx
Tile_DMU
Tile_DMU
TTC-rx
Tile_DMU
Tile_DMU
TTC-rx
s-link lsc
optical fiber
Tile_DMU
Tile_DMU
Tile_DMU
TTC-rx
Tile_DMU
Tile_DMU
TTC-rx
Tile_DMU
Tile_DMU
TTC-rx
Tile_DMU
Tile_DMU
TTC-rx
Many prototypes – test beam tests – earliest
ATLAS subsystem – lots of firsts – production
experience – 2000 boards this year
Instrumentation seminar- 2003-03-20
21
Fysikalisk
Systemteknik
The ATLAS TileCal Digitizer
Task: to digitize pre-amplified PMT-pulses
and to transfer data selected by the L1-trigger
to the higher level triggers.
• 16-bits dynamic range with limited precision
• L1 buffer memory – 2.5 us
• Storage of selected data
• Format data
• send to level 2
• Physical layout
• Noise control
• Radiation tolerance
• Reliability (physical chain – electrical star)
• We also made a optical link with matching
reliability
Instrumentation seminar- 2003-03-20
22
Fysikalisk
Systemteknik
• 16-bits dynamic range with limited precision
• L1 buffer memory – 2.5 us
• Storage of selected data
• Format data
• Send to level 2
• Physical layout
• Noise control
• Radiation tolerance
• Reliability
Radiation tolerant
custom ASIC
- no FPGA
Components Of
The Shelf - COTS
Radiation hard ASIC
10
10
L1 buffer
Event storage
Format and send
ADC
ADC
ADC
ADC
ADC
ADC
Analog part
ADC
ADC
ADC
ADC
ADC
ADC
Hi Lo
gain gain
Digital part
L1a
Trigger and
Timing
Circuit
L1a
L1 buffer
Event storage
Format and send
System clock – level 1 accept
Data till second level trigger
Instrumentation seminar- 2003-03-20
23
Fysikalisk
Systemteknik
Experiences from the digitizer project:
Large scale system design
System aspects – timing and grounding
Reliability
Radiation tolerant design
Production
Even if did not use FPGAs in the Digitizer
we used them extensively when building
prototypes and testbenches.
Instrumentation seminar- 2003-03-20
24
Fysikalisk
Systemteknik
SU – SPECT
Collaboration with Karolinska hosptal
The design of a SPECT camera with an innovative
cylindrical crystal
1 2 . 5
mm
1 2 .5
Le ad
7 2
6 0
Q ua rt z
m m
glass
N aI( T l)
s h i e ld
m m
Co l lim a t o r
h ex
P M -t ub e s
block
72 PMTs around crystal – position determination
via light sharing
Earlier design based on transputers discontinued
Pulse detection + sampling ADCs
Digital pulse processing + digital trigger
Firewire network
Xilinx FPGAs
Texas Instrument DSP – TMS 320 6000 family
Instrumentation seminar- 2003-03-20
25
Fysikalisk
Systemteknik
ICE-CUBE
Collaboration with the astroparticle physics group at SU
Volume 1 km3
1000 m
60 modules/string
1400 m
80 strings
Digital Optical Module (designed by D. Nygren)









Self-triggers on each pulse
Captures waveforms
Time-stamps each pulse
Digitizes waveforms
Performs feature extraction
Buffers data
Responds to Surface DAQ
Set PMT HV, threshold, etc
Noise rate in situ: ≤500 Hz
Photomultiplier
Instrumentation seminar- 2003-03-20
26
Fysikalisk
Systemteknik
ICE-CUBE
The DOM circuit board
2 DOMs share 1 twisted pair for power supply and communication
2 ATWD - 4 channel transient waveform recorder 300 MHz 256 samples
2 channels – hi and lo gain from PMT
Symmetric timing pulses between hub and DOM sampled at
20 MHz 10bits
Supports a higly stable local clock 3.3 ns rms
FPGA and CPU combined in new Altera FPGA
Our part feature extraction
Instrumentation seminar- 2003-03-20
27
Fysikalisk
Systemteknik





ICE-CUBE
Experimental Requirements IceCube
Time resolution:
<5 ns rms
Waveform capture:
>250 MHz - for first 500 ns
~40 MHz
- for 5000 ns
Dynamic Range:
>200 PE / 15 ns
>2000 PE / 5000 ns
Dead-time:
< 1%
OM noise rate:
< 500 Hz (40K in glass sphere)
DOM Pair
20
kB/sec "DOM
HUB"
Proposed IceCube DAQ
Network Architecture
String Subsystem:
60 DOMs
N x 20 kB/sec
N pairs
String
Processor
80 Strings
String LAN
100 BaseT
Total traffic:
0.6 MB/sec
All Hits 0.6 MB/sec
String Coincidence
Lookback Requests
Messages - 170 kB/sec
Fulfill
Lookback Event
Fulfill Lookback Messages
Messages
Event LAN
Builder
0.6 MB/sec
String
100 BaseT
Coincidence
Messages Total traffic:
Built events ~ 1 MB/sec
all event builders)
1.6 MB/sec
Global
Trigger Event Triggers /
Lookback Requests for
Online LAN
all Strings - 0.8 MB/sec
100 BaseT
Satellite
Total traffic:
Offline
SAN
1MB/sec
Data
(Network
Handling
Disk Storage)
Tape
Instrumentation seminar- 2003-03-20
(
--
28
Fysikalisk
Systemteknik
Digital Laser Control
Collaboration with Anders Kastbergs group
Frexghi Habte
Modulation
Absorption
cell
laser
detector
Lock on low
frequency component 0
 lock on maximum
Lock-in
amplifier
Aim: to design a simple laser control that can
manage a large number of units
Our solution: use an FPGA based lock-in module
Asin(wt+f)
x
Asin(2wt+f)-Asinf
Asinf
cos(wt)/2
Cordic algorithm to produce sine and cosine
waveforms + second order Butterworth low pass
Filter (4Hz)
Hardware design based on SPECT module
Instrumentation seminar- 2003-03-20
29
Fysikalisk
Systemteknik
Future
Our involvement in the ATLAS projects
will eventually decrease – the digitizer
during 2004 and the trigger during 2006.
Now they are quite intense
The SPECT camera project should terminate
In its present form this year.
We are participating in a EU application
Coordinated by Anders Brahme at KI. Our
part here would be in the development of a
high resolution whole body positron camera.
Lars Eriksson from KI and CPS would be
partner in this project.
There will surely be other exciting new
projects coming up.
Instrumentation seminar- 2003-03-20
30
Descargar

Utbildning - physto.se