APRIL 2012 BEAM RUN =================== -------------------------------------------------------------------------------- 03.04. 17:00 J.H. ADC channel 58 apparently was bad (memory cell ?). Replaced ADC doughterboard of ADC board 2. 04.04. 13:00 J.H. Adjusted pedestal of the doughterboard replaced yesterday to correct level (1200 counts). -------------------------------------------------------------------------------- 09.04 16:00 E.K. We found new positions of the cryostat center FCAL = 9988 (FCAL100 = 9969, FCAL250 = 0017), old=48 EMEC = 9993 old=63 HEC = 9996 old=70 WARNING: HEC must go in beam first and EMEC must go first out of beam, FCAL can move free -------------------------------------------------------------------------------- 09.04 16:00 Shift crew: Kozelov, Kuhtin, Merkulov, Savin Attenuator set: 30 09.04 21:40 HEC HV ramped up to 1.2 KV 20:30 runs 853,854 unsuccessfull, running with 64 samples calibration settings . -------------------------------------------------------------------------------- 09.04 22:00 J.H. Since the first run was started with calibration parameters loaded while daqctl was still running, please always follow this advice: What to do when switching between Data-acquisition and calibration runs: (1) Quit the daqctl menu . (2) load calibration parameters: "loadcom /home/atlashec/input/calib.par" (3) start calibration shell calsh or other calib procedures. after calibration is done: (4) load normal daq parameters: "loadcom" (5) restart daq menu: "daqctl" If daqctl menu is running while calibration parameters are loaded, DAQ will take data with the calibration parameters, but the daqctl still shows the parameters it had before (unless changed/retyped in the "P" menu)! An indication of running with the calib parameters is if daq takes 4 times the normal rate of ~110 events per spill. If You are going to start a run, but don't know the current state, it is always save to: Quit calsh if it is running quit daqctl if it is running Do "loadcom" to load default running conditions, Then finally start DAQ control "daqctl". -------------------------------------------------------------------------------- 10.04. Nightshift Cheplakov/Huber A.Kozelov working ! 10.04. 00:40 DEN30 foil installed . Leaving it until morning. 01:00 no beam (booster problems) 02:45 beam back, intensity 2e11 -------------------------------------------------------------------------------- 10.04 03:15 J.H. observation: with a spill start/stop timing monitor, I see frequently (~20%) TWO End Of Spill signals. wallclock time indicates second EOS comes few seconds after the first. RF and BT counters are not increased most of the time, i.e. no triggers occured. Is the beam/trigger control disturbed by twice asking for end of spill data ? Watch if this makes problems in analysis program ! Is it caused by bad discriminator/fan-out ? Cure ??? Changed DAQ program to ignore second EOB if no triggers since preceding EOB (tested in run 585). -------------------------------------------------------------------------------- 10.04. 03:45 No Beam 05:10 Beam is back 2e11 10.04. Observation: the Cherenkov data in Cherenkov/bunches/ arrive in a shifted time: e.g. bunches-2012-04-10-07-43-37.dat has creation time 07:55 . The system times on all systems is the same. If this shift is expected (program on the Cherenkov system delays sending to tnf102 ?), then all o.k., otherwise it has to be taken in account when selecting the data. -------------------------------------------------------------------------------- 10.04. 08:00 Shift crew: Kozelov, Peleganchuk 10.04. 10:45 The foil DEN30 was removed from the beam 10.04 16:00 Shift crew: Kozelov, Maslennikov, Schacht Calibrating Cherenkov at low intensity Major problem with ISEG HV power supply. E. Ladygin, L.Kurchaninov, S. Chekulaev and S. Zvyaginzev spend 5 hours (from 13:00 till 18:00) to fix it. Problem was solved by installing two external 24 V power supplies. One HEC HV channel (#1) can not be ramped up. During copying the run 871, the vme computer has got stuck. Recommended reboot procedure (Zepp, please correct !): 1) go to "Minicom Desktop" and issue "minicom" command 2) login as "root" with password "Cazz..." (written on the page) 3) choose Reboot 4) after PPC_Mon> type "boot" 5) wait the finishing of the boot, then quit "minicom" program 6) go back to "VME DAQ Control" window 7) do "telnet vme" 8) login as "atlashec" with password "_hectb008_" 9) issue "loadcom" (may be not necessary) 10) issue "daqctl" to ctart data taking 11 Apr 2012 00:00 E.Kladiva/J.Rutherfoord on shift - Beam is back Continue with pressure dependence measurement of Cherenkov After end of Cherenkov pressure scan start albedo study. Start with half (four) of the iron plates pushed all the way forward on the absorber stand against the retaining screws. After run 877 add four more iron plates. After run 878 add the 29 mm thick poly plate upstream of the stopper screws on the stand. The poly block is in front of the iron plates and upstream of the stopper screws and in contact with these screws. The poly plate is in contact with the upstream iron plate. Run 879 completes the albedo study. Beam is still tightly focused. A.Kozelov requested a Cherenkov run with the beam steered vertically. We removed the absorber plates and turned on the HV to the HEC. HEC HV channels 00, 02, and 03 are set to 1000 V. (Channel 01 is shorted and so the HV is set to zero.) Beam line people were unable to move the beam vertically more then 0,3 (1.0 needed). They eventually gave up. So we moved on to centering the HEC on the beam. We increased the intensity to ~5x10^7, about as high as we should go with the hodoscope in the beam. We'll try the Novosibirsk SEC but it is unlikely we will see anything. We don't see anything on the old SEC. Nothing shows in the Novosibirsk SEC so we turn it off. Take run 880. Savin's on-line analysis shows the HEC is already close to centered horizontally and vertically. But it does appear to be a bit right of the beam center line (looking downstream). We try moving it 2 mm (4 ticks) to the left "out" position. Start run 881. Analyze run 881. It tells us to raise the Cherenkov HV by 0.48 kV. Left-right asymmetry is about the same as in run 880. Move another 2 mm. Start run 882. Lot's of spills where "Failed to get Beam information from 192.168.1.5!" Then it recovers with no intervention. Analysis of run 882 still shows little change. Try moving by 4 mm. Start run 883. Analyze run 883. Getting more symmetric but not there yet. Move HEC out another 4 mm. Start run 884. Analyze run 884. Looks very close. Asymmetry shows we went slightly too far. Try moving in by 0.5 mm. Start run 885. Analyze run 885. There are missing gated cherenkov files starting around spill 22 and Sasha's analysis quits when it can't find these files. Decide to move on without seeing the results of the analysis of this run. Next is to move the EMEC into the beam and align it. We request an access. Move EMEC into the beam at zero on the readout. Looks by eye to be reasonably centered. Start run 886. New center of HEC is 0019 (by DAQ data) In run 886 the on-line analysis failed to finish because it could not find the gated cherenkov file bunches-2012-04-11-07-36-43.dat or one within a few seconds of this. The file from the next two spills are also missing. PROBLEM FIXED (A.Savine) 2012-04-11 A.Savine ATTENTION! Before starting the run, check the Beam Monitor (green display). Spill gate shall start BEFORE the spill to provide time for DAQ to asquire 2-3 empty pedestal events. This is important for the HiLum2012.C online monitor program. DATA WITHOUT PEDESTSAL EVENTS WILL NOT BE ANALYZED. 2012-04-11 Determined max HV: HEC 1200V, EMEC 1200V, FCAL - nominal Beam widest (~4cm fwhm) Runs 905-906 no EMEC signals. Attenuator burned. Replaced Runs 908 ++ HV scan at 1.2e+11 Run 911 almost empty, was started too early, HV was not set yet. To be ignored. 2012-04-12 Eduard, Evgenyj In time of run 916 DAQ finished run with small number of events, copied data and after press "E" crashed with core dump. We restart DAQ and repeat measurement. 2:25 restart programme of the Cherenkov monitoring 2012-04-12 Alexander, Vladimir We continue to go down in intensity, making "One Scan" and changing HV from 800 to 1200 and back for HEC&EMEC. The computer (VME) has stopped during the file transfering (run_937), it was rebooted, run_927 data must be copied manually When started run 938 the message was recorded: EOB 1: 104 events, 104 total - at Thu Apr 12 13:01:35 2012 Beam_Client::socket read error: Needs to be retried; resource temporarily not available Failed to get Beam information from 192.168.1.5! EOB 2: 108 events, 212 total - at Thu Apr 12 13:01:44 2012 2012-04-12 J.Huber after failed file transfer tnf102 gets "mount to NFS server '192.168.1.4' failed: System Error: Connection refused" even after reboot of both systems. No idea what the reason is: verified exportfs on rio2: it does export vs. tnf102. Meanwhile I changed the bin/getrun procedure to use rcp instead of nfs to transfer the data files at end of run. 2012-04-12 Alexei, Peter 17:15 installed 3 foils in beam area 17:26 beam on 18:15 removed 3 foils, average intensity was ~ 10^9, with a short spikes up to 5*10^9, approximated integral 2*10^11 Then having difficulties with getting 2*10^8 : very bad beam structure (almost all intensity is sitting in the spike close to the end, sometimes it is flattening a bit, but no more than up to 0.2 s). Intensity in two separate bunches is very small, sometimes one of them or even both are not visible at all. At ~ 19:20 beam has disappeared (19:40 - no beam from booster). Beam tuning continueing. Cherenkov gives strange numbers - we call Sasha. Restart both Cherenkov programs (as told by Sasha) -> problem fixed. 2012-04-13 00:00 V. Kukhtin, S. Peleganchuk In time of runs 952-956 the beam intensity by Cherenkov had large variation from 2 to 9e+07; bad beam structure 02:40 beam has disappeared (no beam from booster). 03:35 beam on 2012-04-13 08:00 V.Sulin and J.Rutherfoord starting shift Spill structure is very irregular. Also integrated spill intensity jumps around a lot. Run 972 in progress. 2012.04.13 08:10 Ended run 972. Ask for access to change attenuators from 1:1 to 1:3. Sasha Savine changed attenuators. Accelerator plans to take from 09:00 to 10:00 for undisclosed reasons. Too close to this down time to give us the beam back after our access. We lose about 20 minutes. When beam returns we need 1e9 intensity and much better spill structure. Hopefully someone will be in the control room who knows how to tune the xtal extraction. 2012.04.13 17:20 The problems found with the attenuators. Some of them are broken, which revealed in signal absence in EMEC. That is why the runs #973-981 have to be re-done (except 979 ?). 21:00 Run 988 stopped because of HV trip (EMEC ch.0), started a new run#989 instead (invalid) 21:30 Run 990 with NEW EMEC attenuator. Calibration for EMEC not valid. Post-run calibration to be used 22:50 ROOT has crashed when monitoring run_992, re-done it again as run_993. After several attempts it was found (thanks to A.Savin) that the Gate parameters should be changed to have the empty events at the beginning of the spill. We set Delay=168 and Width=1448, it works. 2012-04-14 00:00 A. Maslennikov and P. Schacht starting shift Leonid has excanged the EMEC attenuators, all runs starting with 974(!) up to 982 are bad and have to be redone! The first runs seem to be OK, but starting from 996 EMEC looks rather bad, also ratio to HEC changed! Sasha should check ...! We finish HV scan and will go to lower intensity: attenuators will be changed to 1:10 02:00 go to lower intensity: 1.0e+10, redo run 981, but do full HV scan; Online monitor first events: event display (!) does not work for run number 1000 ?????? 04:00 go to lower intensity: 5.0e+9, redo runs 979-980, but do full HV scan; 05:15 Noticed that the program "Readout", reading the fast Cherenkov, seemed to be blocked (continuously writing "No spill"), the monitoring program "Display" was frozen from 03:56. Restarted "Readout" program after the run 1008. 05:35 replaced attenuators: now 1:3 05:40 go to lower intensity: 2e+9 2012.04.14 08:00 E.Ladygin and J.Rutherfoord on shift Note that two HV channels in the HEC have been off for the entire April 2012 campaign HV channel 1 is shorted and HV channel 0 is disconnected somewhere. 10:15 End of higher intensity running. First ask control room to lower intensity and then take access to install beam scintillators. Also put in film to determine beam profile at low intensity. Probably leave film in the beam to the end of our run on Monday morning. Change attenuators from 1:3 to 1:1. 11:00 Access finished. Try to get beam back at intensity 5e6. 11:15 Start run 1019 at ~5e6 protons/spill. There is a big spike at the beginning of the spill and a big spike at the end of the spill. In between the spill looks very uniform via the gated Cerenkov. 11:40 After run 1019 Savine increased the HV on one of the beam counters 12:01 Looking at Dresden HV current draw we see "spikes" between spills in one channel of three. Currents during the spill appear to be quite measureable, albeit very non-uniform, due to the non-uniform spill. We can compare the currrent as a function of time in the spill with the Cerenkov as a function of time in the spill. 13:23 The "spikes" in the Dresden HV current measurement which occur outside of the spill are probably due to some sort of HV breakdown. Interesting that this is happening at very low intensity as in run 1022. 15:00 Sergey Zvjaginzev repaired two of the 1:30 attenuators intended to be used on the EMEC. One of these two is supposed to be a spare. We are guessing that the EMEC HV "spikes" are the cause of the failure of the attenuators. Sergey replaced the resistors with 0.25 watt resistors. Hopefully these are robust enough to withstand the pulses from the HV "spikes". Peter and Leonid guess that the EMEC HV "Spikes" might be due to a too low liquid argon level. Perhaps the top of the EMEC is above the liquid level. The liquid level drops because of the gas withdrawn to feed the oxygen analyzer. 2012.04.14 16:00 S.Peleganchuk and E.Kladiva on shift 16:30 New argon (1 liter) added in EMEC 17:00 In run with low HV (HEC=500, EMEC=500, FCAL100=40) an intensity 1e+7 no good signals. We return to high HV (HEC=1200, EMEC=1200, FCAL100=100), signals are small in HEC and FCAL100 20:35 Beam is off (power drop down) 21:28 Beam on 23:01 No beam again 2012.04.15 08:00 A. Maslennikov and P. Schacht on shift Large coherent noise (2 MHz) seems to start when the hodoscope/sc. have been moved into the beam. While waiting for beam, Evgenij looks at noise with mstest: noise is at the level of run start!! Conclusion: Coherent pick-up comes with beam ...! Ratio of FCal:(EMEC/HEC) for pick-up is 2, as for normal noise seen previously! 11:13 Illinois oxygen analyzer shows measurements taken last night 0.40 ppm in EMEC cryostat 0.34 ppm in FCal cryostat 0.32 ppm in HEC cryostat 12:28 short tests with hodoscope cables unplugged: signal/HV -> no change, noise stays short test with hodoscope out -> no change when hodoscopes have been moved out, actually the attenuators have been changed from 1:3 to 1:1. Looking back at situation where hodoscope was out and attenuators at 1:1, coherent pick-up noise was the same! Conclusion: pick-up noise was the same all the time! 14:30 Decide to do one more test: unplug the calibration cables -> no change Reconnect calibration cables 14:40 Go to high intensity 2e+11, installed attenuators 1:30 15:45 run 1051: start full HV scan FCAL250 16:00 on shift: Peleganchuk, Huber 17:00 beam off, booster problems. 18:00 Beam on , start run 1053. 18:12 beam off again, end run after 5600 triggers. 19:45 beam comes back ... waiting to reach intensity ... 20:12 Cherenkov VME p4ogram restarted now shows intensity 1.55e+11 20:16 again beam lost with same problems (booster) 20:39 beam back intens 1.62e+11 run 1055 good 21:25 during run 1056 intensity lowered from 1.6e+11 to 1.0e+11 ("radiation too high in kanal"? 21:45 moved Fcal100 into beam , continue full HV scan 22:32 Mini HV scan for Novosibirsk SEC. HV = 100 V 22:36 HV = 50 V 22:40 HV = 0 V 22:44 HV = 150 V 2012.04.16 0:00 Savin, Kladiva on shift 00:00 continue Full scan with intensity 1,0 e+11 02:50 like in last run died attnuators on EMEC (chan 2) 03:00 switch to intensity 3.0 e+8, change attenuators to 1:1, problem with the beam tuning to intensity 2,0 e+8 04:00 mstest shows assymetric noise pattern but we didn't investigate it due time 05:00 rate drop to 7.0 e+7, set back to 2.0 e+8 05:30 beam is out due to vacuum problem 06:15 no beam for long time 07:00 End of "seans", HV off, cryostats out of beam 07:45 start first calibration run with attenuators 1:1, 10:15 new attenuators 1:10 10:53 start new calibration for attenuators 1:10 13:15 new attenuators 1:3 13:46 start new calibration for attenuators 1:3 16:15 new attenuators 1:30 16:39 start new calibration for attenuators 1:30 19:22 calibration finished directory caldata/120416_01 = without attenuators caldata/120416_10 = with attenuators 1:10 caldata/120416_03 = with attenuators 1:3 caldata/120416_30 = with attenuators 1:30