Applications for Synchrotrons.
Sam's Microprobe Analysis Kit, or SMAK for short, is the beginning of a imaging toolkit for x-ray microprobes. It contains the basic functions for:
Importing ASCII fluorescence data and displaying as an image
Support for several colormaps and displays of scale legends
Ability to perform math operations on data channels, including basic addition, subtraction, multiplication, division, as well as smoothing and derivatives.
Production of correlation plots, with the ability to make arbitrary masks for regions of particular interest.
Display of tricolor RGB plots, with a selected channel representing a color.
Ability to combine data channels from multiple files.
Display of raw data from image by selecting data pixels, producing a line plot in x and y directions.
Recall of MCA data if data was saved at collection time.
Ability to rebin data to capture missed peaks.
PyMCA integration for advanced analysis of MCA data.
Pricipal component analysis routines on MCA data. Components can be saved back to image maps. Preliminary routine for using PCA to do EXAFS included as well.
XANES imaging fitting (SEE-XAS or sparse excitation energy XAS) routine to do linear least squares regressions given maps performed at several energies. User enters matrix of standards and gets the fitted proportions as a function of location on the sample.
Data formats from several microprobes have are supported, with more on the way. Current data formats supported at this time are:
SSRL Microscan files (HDF)
SSRL SCANE ASCII files (format for saved files)
SSRL MGScan ASCII files
SSRL Super grid files
GSE-CARS files (ASCII and HDF)
Image and Stack files from AXIS2000 (XIM and NCB)
NSLS-II SRX HDF saved data format
SOLEiL data flyscan formats (NXS)
SOLEIL step scan formats (NXS)
ALS beamline 10.3.2 files
RGB files from Diamond
JPG, TIFF, GIF, BMP image formats
Future ability to include other formats is planned, including possible plug-in type support. A generic ASCII file loader to accommodate other file formats may be provided in the future. The program is still in its VERY early stages of development, but all questions and particularly comments for improvement are welcomed.
There are a few new official documentation sources now. The first is the SSRL beam line wiki page for SMAK. This is always a decent source of information and has some links to the other documents available as well. In particular, there is a discussion here on how to perform semi-quantitative analysis using SMAK with emperical standards.
There is also a slightly out of date (but still useful) word document tutorial that goes through many of the basic functions in SMAK. You can also get the zip file that goes along with this document that contains the tutorial data files.
Finally, there is a pdf document that describes the use of SMAK with the new Xspress3 HDF5 data files that are saved at SSRL containing the raw MCA data from XRF imaging experiments.
Ever wish there was a video tutorial too? Sarah Hayes from the University of Alaska at Fairbanks has made a few of her own YouTube video tutorials for processing data in SMAK. There appears to be more coming down the pipeline, but check them out if you want another reference for data analysis.
At this point downloads of SMAK are in the form of Win32/64 executables (tested through Win7) and now MacOSX (via Wine Support).
If you have a Mac -- PLEASE READ THIS!
With the advent of High Sierra, the Mac DMG installations are no longer working. As an effective workaround, all future releases of SMAK will utilize compatibility with the Wine package for running Windows executables on MacOS and Linux too. For the Mac, you will need to install the latest stable version of WineHQ as well as a recent version of Xquartz from Apple. Once installed you should be able to run SMAK natively as the Windows executable. Note that some installations may only be able to run the Win32 version. We are also working on a stable Python source distribution, but that will come in time!
For Mac installation (with no Wine installation for OS X before High Sierra), please open the dmg file and drag the application bundle to you "Applications" folder. It will not run from the .dmg file or any other location on your filesystem. There may still be issues with running the application - if it crashes because it cannot find "tile", download this zip file and install the folder to your "/System/Library/Tcl" directory. On some Macs the directory may also be "/Library/Tcl". You will likely need Administrator privs to do this. If you have MacOS > 10.6 (i.e. Mountain Lion or Mavericks) you may also need to install the new Apple flavor of X11, xQuartz. This should install into your Applications-Utilities folder. Many users have reported that they need to duplicate this program to also be titled X11 (so... select "xQuartz" in a Finder window, copy/paste, rename the new file to "X11").
Starting with Mac OSX 10.11 (El Capitan), Apple has started using "System Integretity Protection". If you have this, you will NOT be able to follow the directions above to install the "tile" folder and SIXPACK will not be able to run. However, if you choose, you may disable the SIP to perfrom the "tile" installation, and then enable SIP again afterwards. This link describes the proces that you need to follow.
If you have found that SMAK has suddenly stopped working or fails to run after all of this... You may have upgraded your system as of late. There is a known apparent error (that manifests itself as a "gestalt" version error in the console) with upgrading the Apple OS. While it is unclear what the best way to resolve this, some Google rumors suggest reinstalling your OS. From my personal experience of updating from Mavericks to Sierra, I was able to get everything working by (1) reinstalling the OS, (2) reinstalling the Tcl Tile package (as noted above), (3) installing or reinstalling xQuartz. Please let me know if these things work or don't work.
Source code in Python source for Linux, Mac or Unix machines will be provided soon. As of now, it is still relatively untested on non-win platforms. Any feedback on the use of the code on other platforms is greatly appreciated.
Download v1.64 Windows 32 bit executable in .zip file.
Download v1.50 Windows 32 bit executable in .zip file.
Download v1.50 Windows 64 bit executable in .zip file.
Archives of previous versions
Download v1.40 Windows executable in .zip file.
Download v1.40 MacOSX disk image in .dmg file.
Download v1.30 Windows executable in .zip file.
Download v1.30 MacOSX disk image in .dmg file.
Download v1.20 Windows executable in .zip file.
Download v1.20 MacOSX disk image in .dmg file.
Download v1.10 Windows executable in .zip file.
Download v1.10 MacOSX disk image in .dmg file.
Download v1.00 Windows executable in .zip file.
Download v1.00 MacOSX disk image in .dmg file.
Download v0.52 Windows executable in .zip file.
SMAK is Open Source Software. See a copy of the license here.
The interface portions of SMAK were developed by Sam Webb at SSRL, the Stanford Synchrotron Radiation Laboratory. Thanks to Graham George for developing the SCANE program for imaging at SSRL and to Martin George for enormous computing and programming support.
10/13/19: Version 1.64: Major new release, (although there will be a delay to get the 64-bit Windows version). Fixed several of the remaining errors that existed in the previous version(s). Expanded additional support for SOLEIL data formats. Added visualization for PCA components and added an ability to limit the number of PCA components found, so that you are not limited such that the number of components is forced to the number of data channels. New Simplex Volume Method (SiVM) for end-member detection added, using an adapted method of Matthias Alfeld. Support for full field XANES imaging analysis included, and the ability to export clustered spectra from a K-means cluster map.
10/11/18: Version 1.50: Major new release, including a 64-bit Windows mode. Note that all Mac support will be through the use of the Wine program. Added 32-bit TIFF support, as well as expanding SOLEIL data formats. Updated references to plotting libraries, so the graph windows may look a little different. Added support for SSRL UV-vis spectrometers as MCA files, with the ability to integrate raw UV-vis data to XYZ/RGB/LAB color coordinates. New clustering algorithms added Numerous bug fixes added along the way.
8/12/16: Version 1.40: Minor fix in the MCA fitting. Error appeared due to memory contratints for large, ok even moderate sized data. Fixed now.
3/28/16: Version 1.30: Major release! Minor fixes from last release. New functionality includes usage of PyMCA routines to do fluorescence fitting to full MCA spectra (must be in SMAK HDF format). New MCA fitting interface includes peak fitting, matrix and absorber effects, as well as the ablity to define different matricies at different pixel positoins, for instance based on Kmeans clustering results! Included material database to GUI paramters as well. Can fit a single spectrum, a zoomed area of the map, or the entire map area. Can also apply gaussian spatial filter averaging to the MCA spectra before performing the fit. The interface is still experimenta and will probably evolve in the future, but its a good start to include MCA fitting.
2/29/16: Version 1.20: Major release. So much in here, I'm sure to forget a lot of it. Updated the file formats to support several other image formats. Text files are read with a universal newline interpretation, so files saved on a PC can be read by Mac/Unix and vice versa without having to convert the newline characters. The PCA selection now has a large series of multivariate analysis options, including "sPCA" - Principal Compent Analysis, "CCIPCA" - Candid Covariance-free Incremental Principal Component Analysis (the previous standard algorithm), "FA" - Factor Analysis, "NMF" - Non-Negative Matrix Factorization, "FastICA" - Fast Independent Component Analysis , "Kmeans" - K-means Clustering. File imports will attempt to read the energy in the data file and automatically append that to the column name. File menu has a "Rapid Import" option, where the first selection of channels is applied to all files and channel names are automatically calculated or incremented. The XANES fitting analysis option has a dialog to import the appropriate energy values from saved, 2-coilumn mu data files. Added "Invert" to the "Advanced Filter" selection to easily invert several channels without the "Map Math" window. Plot markers section has option to save marker data as an XASSCAN queue so points that are picked from maps can be exported to the XAS data collector. There's probably a lot of ther things too...
7/25/14: Version 1.10: Major re-release to fix many of the long standing issues. Added too many new features to comprehend. Supports use of SMAK friendly HDF files and support of the Xspress3 HDF MCA files. This also helps loading large files. All users are encouraged to change the old SMAK .mca files to .hdf files so that MCA usage becomes extremely feasible!
7/14/12: Version 1.01: Quick bug fixes. Some minor fixes. Added a fix to remove excess calculations for user-defined masked statistics.
7/9/12: Version 1.00: Major re-release! Including MacOSX flavor! New code removed Windows dependencies on the code, so expect to see a source release as well as a Linux version soon. The toolkit now uses the OpenCV package for many of the image processing and filtering, so the code is much faster. Adding more sophisticated thresholding and filtering routines, including Fourier transforms for noise removal. Can create manual masks and do mask addition operations for defining areas of interest. This is of use mostly in new statistical analysis package which will provide detail inter-map analysis of ROIs, either by manual detection or discrete particles through thresholding analysis. Will perform box-and-whisker type plots, histograms, and do ANOVA statistics between ROIs. Masked and zoomed areas are now used in the PCA and data compression and data summary displays. Moment analysis also uses masked and zoomed areas. Correlation plot raw data can be exported to the clipboard. All plot data in general (including the statistical and histogram data) can also be exported as text. Several other small things that I'm sure I've forgotten! In the new release, the computed tomography codes are being reworked and may be broken. Use the older version of SMAK in the meantime until the fix is posted.
10/28/10: Version 0.52: More new features! Masking on the image directly works correctly (the shift left click can define an ROI or area for MCA examination or both). Data summary now also lists mean and standard deviations for the view/mask being shown. Correlation plots can give a best fit value and r-square correlation. Option to convert the ASCII .mca files to binary for faster access if routine rebinning or examination is required. This can speed up the process by a factor of almost 4. Can make movie animations from the MCA data. Working on several other file formats as well as a whole host of updated features. Stay tuned!
02/19/10: Version 0.51: Many new updates yet again! Separated math operations into two menu bar items – Process and Analyze. Added set of dialogs to perform simple quantification. Does not account for thickness effects or matrix effects as of yet. Added advanced filtering, including mean, median, min, max, blur, and unsharp filters. Also added a custom kernel filter. Histogram function added. Added new plot marker shapes and a scrollbar in the plot marker window. While zoomed, the keyboard arrow keys will pan your view. The summary outputs, correlation plots, etc, are all linked to this view and will update as well. Option added to perform line fits (linear, gaussian or both) to line profiles created with the ALT shortcut. IND function included in the PCA output (text in the DOS window) when performed on channel data. Can shift pixels in individual rows. Radial profile maker added. Advanced stats including moment analysis added. Can make a mask, or define a region of interest, on the map image directly using the SHIFT shortcut. Whew. More to come soon as well I hope…
09/21/09: Same version, but fixed an error with the new colormaps data file.
08/28/09: Version 0.50: Many updates again! Added expanded colormaps to plot displays. Added the ability to place “markers” (i.e. square, circles, text) at given coordinates. These can be saved etc. Useful for marking where XAS points, etc were measured. Fixed issues with multiple zooms in the view. SMAK can now write a SIXPACK deadtime file. Fixed issues with correlation plots when deadtime corrections are being made. Can save image as hi-res TIFF. Can do a “remove edges” function from the display window (right click) to remove the edge from the display. Good for quantification summaries. Can also display side scale axes (cannot be saved yet). MCA over a selected region can now be chosen as a sum (as it used to be) or the average (divides by the number of points). PCA can now be done on individual channels (the new default) rather than MCA spectra. Great for multiple energy maps. Fixed errors present in the tomography >backprojection filters. Whew. Will add support for auto-reading ICP-MS inputs in next release.
04/15/09: Version 0.47: Many updates. Added a non-negative least squares fitting routine for the XANES imaging/fitting routine. Actually added two. The first one “A” should be the most efficient. Added a stereo-image maker. Fixed additional errors in ALS format with the addition of the subtitle field. Fixed issues with jpg saves of the display without the scalebar present. Added abilities to “stitch” images together. Added a deadtime math operation. Fixed many of the issues with underscores in channel names – best to avoid them however if you want to make tricolor plots! Added ability to create data files from the saved MCA data. Can also import XRD data created from the ADM saved integration format. More to come of course, but these will do for a while!
03/03/09: Version 0.46: Bux fixes for some of the large file format issues. Added support for APS MRCAT files.
02/04/09: Version 0.45: Big jump in revision number for “official” posting, since there were several small unofficial releases. Added better support for large files and the ability to read MCAs etc for large file formats. Fixed some issues with the display of large files (again!). Support added for deconvolving beam size from the data. Error analysis added. Support for RGB files from Diamond added. Probably a few other things too. More updates coming soon as well!
04/28/08: Version 0.39: Few minor bug fixes. Fixed issues with screen sizes of images (again). Added new support for tomography!
01/25/08: Version 0.38: Few minor bug fixes. Also added feature which warns users that unsaved work is present and should be saved before closing files or the program.
10/25/07: Version 0.37: Added the proper HDF tools and DLLs for HDF support of BL6-2 and NSLS files. Also added preliminary support for TXM files from SSRL BL6-2.
9/08/07: Version 0.36: Added support for SSRL BL6-2 super data files. Scale bars now user moveable. Fixed apparent issues in the PCA analysis with large files. Probably a few things I’m forgetting too…
6/27/07: Version 0.34: Fixed an issue with the scale bar not saving in the tricolor plots.
6/18/07: Version 0.33: Fixed a reported bug with the sum MCA area functionality. Also upgraded many of the DLLs so that the displays are better (hopefully). Fixed the linked scalebar issues as well.
5/30/07: Version 0.32: Fixed a reported bug regarding saving MCA traces to files and the windows clipboard.
4/30/07: Version 0.31: Fixed a DLL error in last release. Zoom and >scalebar functions now work in tricolor maps as well. Both normal maps and tricolor maps share the same zoom parameters. Correlation plots now show the correlations in the zoomed in area. Option to switch correlation plots colors to black and white for screen capture friendly data. Can use the “Set MCA file lines” and “Split MCA files” to separate MCA data collection files taken at multiple energies – i.e. to split up MCA files from data collected above and below a particular absorption edge.
4/24/07: Version 0.30: Fixed some math errors that arose in the previous version (0.28 and 0.29). Added a zoom function (drag a rectangle with the control button held down) and an arbitrary cross section line viewer (use the ALT key). Added some preliminary (fairly untested) ability to fix MCA files with gaps. >Scalebar option added to viewer.
2/16/07: Version 0.28: Added multiple window option and label of the data channel at the top of the display window. A right click on the channel list will open an additional display window of the chosen. This allows viewing of multiple channels simultaneously. The additional windows are destroyed when closed, not hidden.
2/15/07: Version 0.27: Added ability to import single channel data from JPG, GIF, TIFF, and BMP image formats.
1/24/07: Version 0.26: Added routine to sum multiple full spectra points. Use the shift key on the display image to select a region, double click to finish – if a MCA file is defined, the sum of the regions MCA will be shown. Fixed a bug which caused trouble with incomplete MCA files. Known issues with flipping the axes in the viewer to be resolved soon.
10/19/06: Version 0.25: Added progress indicators for PCA file loading and computing progress. Added fast search routine for MCA retrival to reduce wait times. Fixed an occasionally bug that would report the wrong pixel coordinate on the display readouts. Added a XANES fitting routine for doing a least-squares regression of multiple energies to calculate species proportions.
8/8/06: Version 0.23: Screen size issues fixed with the option to change the minimum displayed size of the display. Added functions to average across the axes. PCA analysis and varimax rotation of MCA map data has been added and may be updated in the near future (still need to add a few of the save options).
7/30/06: Version 0.22: Fixed small errors in routines for loading SUPER files and SCANE files. References to pixel maps also improved. Screen size issues fixed again, this time I think for good. Added MCA viewing and re-integration support. Deadtime corrections now present.
7/24/06: Version 0.21: Added support for PNC-CAT data types. Also added corrected issues with data windows for small screens (I hope), added scroll bars for the tri-color plots selection, and the ability to flip the data in x-y coordinates. Adding functionality for data collected at SSRL to view MCA traces at individual points. Next update will have the full functionality to view and data in real time.
6/5/06: Version 0.20: Added support to sum multi-element detector summation and included nifty new logo.
5/22/06: Version 0.19: First public release of SMAK.