Sam's X-rays
2-3 Troubleshooting
1. There’s no beam
1. Is the shutter open?
-
If no, then open the shutter.
-
If yes then move to #2.
2. Has there been a beam dump?
-
If yes, then wait until beam comes back.
3. If no, is there signal in I0?
-
If no then: did you change the mono energy by a large amount (1000's of eV) without running the macro that your beamline scientist set up for you? If you did not run the macro, then you should run the macro. If you did run the macro and there still is no beam, then contact the beamline scientists on Slack.
-
You can also check beam tuning:
-
In microXAS GUI, select "Scan Motor"
-
As your detector, select "I0"
-
As your motor, select "TUNE"
-
Enter scan range (relative): -1 to 1
-
Enter step size: 0.1
-
Hit "Scan"
-
Select the maximum using the cursor, and then hit "Move"
-
-
If you check TUNE, you may also want to make sure the TABLE motor is in the correct position i.e., Do a TABLE scan
-
For this, you need a clear path for the beam to I1 (so move the sample out of the way)
-
In microXAS GUI, select "Scan Motor"
-
As your detector, select "I0"
-
As your motor, select "TABLE"
-
Enter scan range (relative): -1 to 1
-
Enter step size: 0.1
-
Hit "Scan"
-
Select the maximum using the cursor, and then hit "Move"
-
NEVER SCAN TABLE MORE THAN -2 to 2. If you can't find the beam after attempting optimization, contact your beamline scientists on Slack.
-
4. If there is beam in I0 but still no beam on the sample: the optic may have gotten misaligned somehow. Contact the beam line scientists on Slack.
2. I cannot see my sample in Microscan/Microscan video feed frozen
Try closing and restarting Microscan. Close Microscan via the command prompt window (look for the command icon on the bottom tool bar- it looks like a black square. Then hover your mouse over it to see all the command windows and select the one that has the microscan icon). If there is more than one Microscan command prompt, close them all. Multiple command prompts confuse Microscan.
-
If the video feed is still frozen or black, you will need to restart the camera server. This will not interrupt data collection.
-
Close Microscan
-
Locate the remote desktop window (the same window as the laser read out values). Try closing the window ‘camera server’ by the red X. If it closes, reopen camera server from the desktop. Jump to last step.
-
If the camera server window does not close by the red X, power cycle the camera on the PDU (GUI with outlets) by checking ‘off’. You should see the camera server window in the remote desktop close when you turn it off in the PDU.
-
Turn the camera back on in the PDU. You should notice the camera cycle through zooming on the ‘live’ video feed on the monitor to your upper left.
-
Open the camera server from the desktop of the remote desktop.
-
Open microscan, click F1 and camera will refocus.
-
-
Alternatively, the video viewing software might need to be restarted.
-
Find the VNC icon on the bottom left side of the screen (pinned to the task bar) and open. This opens a remote connection to the computer running the video viewing software
-
When prompted for a password, enter bl23cam
-
Open a python3 window (bottom right on the task bar):
-
-
-
-
-
Type "python vidserver.py"
-
Open a second python3 window (you can right click on the icon and select open)
-
Type "python vidclient.py"
-
This opens a display of the videofeed. You can now close the remote connection.
-
On the main PC, open a cmd promt and also type python vidclient.py
-
A mirrored video feed should now appear with a label Receiving Video
-
3. Microscan is unresponsive/Sample stage will not move
-
Try closing and restarting Microscan. Close Microscan via the command prompt window. If there is more than one Microscan command prompt, close them all. Multiple command prompts can confuse Microscan.
-
If Microscan opens and is still not responsive or doesn't open, this could be due to a few things including:
-
The system still thinks there is an active scan still running and won't let you move the stages. This is by design.
-
One or more of the sample stage axes have disabled. Did you drive out of range? (past 12 or −12)
-
The XPS (stage controller) needs a reboot. A way to double check this is that you close and try to restart Data Collect (Master Data queue) and that doesn't start either AND the Newport webpage will not load.
Solution to 1:
Check the Master Data Queue window. What is the data collection status light showing? If the color is green or orange it means Master Data Queue is still doing something and hasn't released control back to Microscan. Also check the bottom left of the window to see if there is a message like "Data queue active". If this is the case, try the following:
-
Try hitting the "Abort Scan" button.
-
You can also clear errors and reset the queue under queue controls. Try doing both.
-
If the above two options to do change the light status to gray, try closing Master Data Queue program. Use the command prompt window called "DataCollect" to ensure the program fully closes. To reopen, use the desktop icon called "Data Queue". If Master Data Queue does not reopen, go to solution 3.
Solution to 2:
READ THIS PROCESS IN FULL BEFORE ATTEMPTING! There is an extreme likelihood of the wheel hitting either the optics or the detector. If you have the single (rock) sample holder on or thick samples you should remove them before completing this process.
First, check the status of the stage motors:
-
Double click the "XPS-D" shortcut on the desktop. If you cannot find the shortcut use a browser such as internet explorer and in the URL field enter the IP address: 192.168.1.245.
-
Login with:
Username: Administrator
Password: Administrator
If you get no response from this window then consult Solution 3 below.
-
On the command bar, click FRONT PANEL. Then click MOVE.
-
This should bring up the Newport status for each axis of motion for the stage. The functioning state of the buttons under the “Action” column should read “Disable” for Y and Z and “Disable slave” for the X axis, if the axes are working as they should be. If an axis has failed the button will read "Enable" or “Initialize”.
-
If a single axis reads as "Enable" click that button. Things should work again now. But, if one or more axes read "Initialize" then....
-
To make sure you do not hit the optic (causing > $100,000 of damage and disabling the beamline for ~6 months):
-
Move the optic to the through hole using the Hex GUI and move to S0 (the phosphor)
-
Alternatively, you can remove the sample wheel (you will lose your exact sample positions if you do this)
-
-
To make sure you do not hit the detector:
-
Move to S0 and remove the sample/sample wheel. This ensures no contact with the detector or optic. OR:
-
In uXAS GUI, click “Move Motor”.
-
Scroll down to “DETECTOR” and click it. It should show the current position. Make note of it.
-
In the “Move To” box type in a LARGER number than in the current position. Say 15.
-
Click “Move” button and the detector should move away from the wheel. You can confirm this in the webcam.
-
-
Click each “Initialize” button in the Newport web browser front panel (including SX and SY). The buttons should change to “Home”.
-
Click the “Home” button on the X AXIS FIRST! (If you do Y first the wheel will drive with force towards the optics.) The axis should return to the zero home position and the status should now read “Disable”.
-
Click the “Home” button on the Y, Z and Phi axes, but do not home the SX and SY axes (if you home SX and SY you will move the position of the sample camera and lose sight of your sample). The Y, Z, and PHI axes should return to the zero home position and the status should now read “Disable”. Note the sample wheel will return to S0.
-
Locate the command prompt window that has a last line ending "..../newportStages". Hit the up arrow on your keyboard and you should see a command appear "python initializeXPS_MA". Hit enter.
-
Go back to the Newport browser page and you should see that all motors should now show "Disable" and the X axis should now read “Disable slave”.
-
Close and reopen Microscan.
-
Remember this changes the focus point of the samples in the X-ray beam, so you will need to refocus on your sample using the red arrow buttons and the laser read out. Therefore, if you had already collected a map and were going to pick XAS spots on that map, the spots are likely no longer precise. You may consider collecting the map again.
-
Don't forget to return the optics to M5 or M1.
Solution to 3:
Check with a BL scientist to confirm whether the XPS needs to be power-cycled. Locate the web browser (internet explorer) for the Newport stages. If you cannot find the webpage or desktop shortcut, use a browser such as internet Edge and in the URL field enter this IP address: 192.168.1.254.
-
Try logging in with:
Username: Administrator
Password: Administrator
Permission: Administrator
-
If you successfully log in then:
-
Select "IPManagement" from the top menu.
-
There is a red "Reboot" button at the bottom of this page. Click that and wait a few minutes for the unit to reboot.
-
Try refreshing the page. Once the login page appears, follow the steps set out in solution 2 above from step 4.
-
If the Newport browser page is not reachable then we need to power cycle the stage controller box in the hutch called a Newport XPS.
-
The stage controller is a beige box, located on the hutch floor on the right hand side
-
Find the power switch next to where the power cable is plugged in. Turn the switch off and back on again after a few seconds.
-
Refer to the steps in "Solution to 2" above from step 1
4. My maps look streaky/no map data is showing
When Microscan moves to a new mapping region, or you start a new scan, Microscan does not reset the live mapping windows. The maps can look streaky or show no data at all.
Solution 1:
-
Close and reopen Microscan. Remember to close Microscan via the command prompt window.
-
Make sure you are on F1 so the video feed is in focus.
-
Click on the map that is currently collecting in Master Data Queue, it will highlight in blue.
-
Go back to Microscan and click ‘Get curr in queue’. This option imports the map information from Master Data Queue for the current mapping region. Check ‘show current region’ to see the red box on the microscan video feed.
-
Try opening map windows now to see your data collecting.
-
Open your data in SMAK and verify that you have data
Solution 2:
If you have done solution 1 and no data appears in SMAK (i.e. when you load your map, all element channels show zero counts across all pixels), then abort your data collection and go to Xmap GUI
In Xmap GUI, from the "detector" menu, select "Run zebra reset batch file"
Click "start" to verify Xmap is collecting
Restart the data queue
Solution 3:
If you still do not see any data, see Problem 8, solution 2.
5. The optic is frozen – I cannot move between M5, M1 and through
The optic motion can freeze from time to time, there isn’t necessarily something you did to cause this. The solution to this is similar to fixing the stage with a frozen axis.
-
Locate the web browser for the XPS-Hex.
-
If not already logged in, login with:
-
Username: Administrator
-
Password: Administrator
-
-
Click Front Panel, then move.
-
If an axis is not working, it will read “Enable” or “Initialize”
-
Click the button(s) that real “Enable” or “Initialize” until they read disable.
-
Now move to the desired optic position in the Hex GUI. You may want to close and reopen Hex GUI too.
6. I cannot see my XAS data or my data looks really crappy
This is usually due to:
1. Having the wrong display parameters selected for the data you are collecting.
2. The plotter window having the zoom function or the vertical cursor still activated somewhere.
Solution to 1:
-
For fluorescence data being collected in SCA1, you need FF/I0 selected in the Std Plots tab.
-
For fluorescence data being collected in SCA2, 3 or 4, you need to select Custom in the Std Plots tab. Then in the Custom tab select Requested Energy as the numerator, and 1 as the denominator. Then you would select the appropriate SCA channel as the numerator (e.g. SCA2.1), and I0 as the denominator in the Y-axis group area.
-
For transmission data you should have the appropriate ion chambers selected, usually I1, either in Std Plots or in the Custom tabs.
-
You may have the first derivative selected. Go to the Math tab and just click the "None"radio button.
Solution to 2:
The display control buttons under the main plotter panel act as toggle switches i.e. one click to turn them on, one click to turn them off. As such they can be inadvertently left on causing the default auto scaling function of the plotter window to be overridden.
When the zoom rectangle or the pan scan buttons (highlight by the red box below) are activated the words “zoom rect” or “pan scan” will be displayed to the right of the icons (green box below). If this is the case, click the buttons again to turn them off. Your spectra should now appear and be auto scaling (if a spectrum is currently being collected).
If this doesn’t work then the vertical cursor may be still active somewhere. In the bottom left corner of the XASplotter window (purple box below) there is some small blue text that will either say “Marker at Nonee” (then it is already off) or “Marker at energy”. If it says a number then it is active somewhere so click just off the plot axes to deactivate it. This area should now read “Marker at Nonee” (yes it’s a typo).
7. The ion chambers show no reading (i.e. 0) or the numbers are not updating
1. Is there beam?
-
If you can see beam on the phosphor, restart the ion chamber GUI (DVMs). The ion chamber readings should update.
-
If no, go to ‘There is no beam’ for solutions.
8. I have clicked ‘start queue’ or ‘start scan’ and nothing happens. OR the beamline has stopped collecting data mid-scan.
1. Is there beam?
-
If no, go to ‘There’s no beam’ and follow steps.
-
If yes, move to #2
2. Check the XSPRESS3 control window (usually in the upper right of the desktop,). Is the text on the GUI all green?
-
If yes, move to #3
-
If no, and the boxes are white:
-
An IOC restart is required. To restart IOC, look for the MobaXterm window- it will be in a black icon at the bottom left of the screen (pinned to the task bar). If you hover the mouse over this icon it will read an IP address (typically 192.168.1.60) followed by "(xspress3)"
-
There will be several tabs open. Scroll to the top of each tab until you find the command "python epicsrelay.py".
-
Hit ctrl+c to terminate this process.
-
Hit the up arrow, you should see "python epicsrelay.py". Hit enter.
-
Next, scroll to the top of the remaining open tabs until you find the command ‘xspress3-ioc.sh’. Once you have found this tab, perform the restart, as follows:
-
To ensure the previous ioc process was terminated, press ctrl + c. This should pop up a $ on a new line. This signals the previous process was terminated
-
Hit the up arrow, you should see xspress3-ioc.sh’ and hit enter.
-
The XSPRESS3 control window should turn from white to green writing
-
-
You will need to close and reopen XmapGUI. Once opened, click one of the detector channels and press start. If you get an error, try closing and reopening XmapGUI again. If you still do not see counts try closing and reopening XmapGUI and running the zebra batch file: this is under the "detector" menu --> "Run reset Zebra batch file".
-
If you see an MCA trace, make sure your elements are still selected and highlighted by the colored bars. You can double check by going to the periodic table interface and making sure the same elements are selected.
-
You may also need to close and reopen Master Data Queue, Microscan and microXAS (the data queue must be opened to open microscan or microXAS). This should not be necessary, but sometimes it's helpful to restart Master Data Queue.
-
A description of how to restart the full suite of detector software is given in #13. It is very unlikely you will need to do this (most of the time, only one of the programs - xspress3.ioc - needs to be restarted as per the directions in this section).
-
3. Are you within the stage limits (absolute limits are +12 to -12 but we suggest going to a max of +10 to -10 due to the way in which the stage moves to start each line of a map).
-
If no, or you are unsure, check the co-ordinates in Microscan on the left hand side. And, check the Newport webpage to see is any stage axes have disabled. Then, see ‘Microscan is unresponsive/Sample stage will not move’ solution 2.
-
If yes, move to #4
4. Do you see a "low limit" in the command prompt for the Master Data Queue (dataserver)?
-
If yes, does I0 read < 0.1 in the DVM?
-
I0 being < 0.1 is usually why you get this error. Optimization of beam and mirror pitch feedback are important here, you may wish to ask for help in slack unless you have gone over optimization parameters with BL staff.
-
If the beam is well optimized and MPFB is set (between 40-60%) then increase the gain on i0. The value for I0 should increase in the DVM.
-
Restart the queue
-
-
If no, or are unsure, check in with BL staff or move to #5.
5. Did you try to restart the queue in Master Data Queue, or restart Master Data Queue?
-
If no, abort the queue. Reset the queue and clear errors under ‘errors’ tab. Restart the queue. And/or restart the data queue and attempt to restart.
9. I restarted Microscan, but it will not open
This can happen if the stage has not properly been reset (i.e. a stage axis limit was hit). Go to:
and follow solutions 2 and 3.
This can also happen if microscan has lost contact with the sample camera/video feed. Go to:
This can also happen if something else is wrong....contact you beamline staff if prior solutions don't help.
10. The mono is not moving as it should, or I see #ERROR# in microscan/microXAS/data server
Sometimes the ICS server (the server that interconnects all the beam line electronics) disconnects. This sometimes presents itself as an error (#ERROR#) in uXAS and/or MICROSCAN.
You will need to restart ICS:
1. Close DataServer, microXAS, and microscan
2. Go to the remote desktop app in the task bar. The BL scientist who set you up should have already set up the appropriate connection, 192.168.1.1. (If there is no connection, you will need to contact the BL scientists to get this as we cannot put the log in credentials on this page for security reasons. If you cannot contact the BL scientists then you will just have to wait until we respond).
3.Open a command prompt or terminal window.
4. In the terminal type “ics_cpc” (without the quotes) and then press enter.
5. Then type “4” enter.
6. Agree to “default” by pressing enter .
7. Type “Y” enter to select yes you really want to restart.
8. You may wait up to a minute or so until ICS shuts down.
9. Now type “3”.
10. Agree to “default” by pressing enter.
11. Several seconds should elapse and ICS will be restarted.
12. Restart DataServer, microXAS, and microscan
11. I do not see any counts in XmapGUI
1. Have you selected one of the circles on the left of the GUI that represent the channels of the detector?
-
If no, select any channel and count again
-
If yes:
-
Try running the zebra batch file. There is usually a command prompt open to run the zebra batch file, locate it and press the up arrow, hit enter.
-
If there is no command prompt, check file explorer to see if it is open in the correct location for the zebra batch file. If it is, double click on the zebra batch file. The command prompt will close when it if finished running.
-
Try counting in XmapGUI again. You may want to close and reopen XmapGUI.
-
2. Check the XSPRESS3 window in the top right of the desktop;
-
Are the words white? If so, this is an IOC restart. See:
-
If the words are green, do they indicate that the XSPRESS3 thinks that data is still collecting?
-
If yes; check that no data are collecting still. If you confirm that no data is collecting and the XSPRESS3 is wrong, press stop in the XSPRESS3 GUI on the left hand side. Try counting in XmapGUI.
-
If this does not fix the problem, you will likely need to follow the IOC restart. See:
-
12. I cannot see my data in the T drive
Data are automatically backed up to two locations; our master data backup and to your groups T-drive. If you cannot see your data in the T-Drive:
1. Double check the spelling of the drive. We will sometimes designate the “T” drive as any other letter. Solution:
-
Look for the data backup GUIs. One GUI will be the back up to your “T” drive. You will see the drive location for data back up at the start of each line in the back up GUI. This could be almost any other letter (but not “Z”! the “Z” drive is almost always the master data backup).
-
If you still cannot locate your data:
-
Find the data backup GUIs. Press stop and start.
-
Look in your data folders for data
13. ALERTS format
To receive notifications from the beamline about errors, beam dumps or finished queues, add you email address or your phone number into the bottom right text box of the master data queue. For phone numbers, add as:
-
Verizon - 10digitphonenumber@vtext.com
-
T-Mobile - 10digitphonenumber@tmomail.net
14. Starting the detector software from scratch
-
MobaXterm is a remote desktop application that allows you to remote into the Linux PC on which we run the detector software from the beamline control PC.
-
Open MobaXterm (if not already open)
-
If Moba Xterm is already open, close the programs and the windows that are already open by hitting cntl-C, and then at the prompt type "logout", which will close the window.
-
Click on the session tab, and you will see a list of "Saved Sessions". Select 192.168.1.60 (xspress3). Open a total of 5 tabs by double clicking the IP address
-
In the first tab, scroll up or type the following:
-
cd mnt/ext8
-
python epicsrelay.py
-
-
In second tab scroll up or type the following:
-
source ./exportZebra.sh
-
hit enter
-
In the same tab type zebra-ioc.sh
-
hit enter
-
-
In third tab scroll up or type:
-
- source ./exportZebra.sh
-
hit enter
-
In the same tab type zebra-edm.sh
-
hit enter
-
-
In the fourth tab scroll up or type:
-
xspress. source ./exportXsp.sh
-
hit enter
-
In the same tab type xspress3-ioc.sh
-
hit enter
-
-
In the fifth tab scroll up or type:
-
source ./exportXsp.sh
-
hit enter
-
in the same tab type xspress3-edm.sh
-
hit enter
-
-
The commands zebra-edm.sh and xspress3-edm.sh will open new graphical user interfaces that allow you to interact with the detector software. Users typically do not need to interact with these pieces of software. Staff members will use them to diagnose problems. It will say "connected" or "system connected" somewhere on the interfaces (pictured above) when first opened. If it says "not connected" or "system not connected" you're out of luck and will have to contact your beamline scientist! Note there are a couple other windows that also get opened when you run the "-edm.sh" commands. Just minimize these- you can ignore them.
-
After restarting the zebra and xspress3, you will need to restart Xmap GUI. Double click the Xmap GUI desktop icon. When it opens, go to the "detector" menu and select "Run zebra reset batch file". Then, click the circle with the "1" in it and press start. If beam is on and your sample is in, then you will see a fluorescence spectrum displayed on the plot and counts listed under ICR and SCA1 etc. If you do not see this, or if the program hangs, you should close it and try again (just click the "x" in the right hand corner to close).
MobaXterm Interface
Xspress3-edm.sh
zebra-edm.sh
