Journal notes, Harmonic Inversion and FDM
GELLER Labs "Backyard Science"
Thoughts on a proton precession magnetometer design - a Proton Magnetometer Project
The goal of this project is a low cost high performance proton magnetometer (a digital magnetometer) kit for amateur scientists to be able to accurately measure and monitor changes in the Earth's total magnetic F field and to observe geomagnetic storms. There is a regular daily (diurnal) variation in the Earth's magnetic field. During events related to solar activity, there can be sudden changes in the field (such as a sudden impulse) as well as large excursions in the field which can be more than ten times the regular diurnal variation caused by magnetic storms.
(be sure to hit refresh to pick up our latest changes and entries)
Tuesday, January 5, 2010
Ran overnight with only 4 taps on the digital FIR filter, PDF, txt, USGS PDF.
Then, at a normal diurnal minimum, backed the Subaru wagon out for a ramp test (field at 30 feet is depressed by the wagon). Buneman fails yet again (PDF, txt) with a big gap. So, it is probably time to abandon Buneman, but development on LabView was remarkably easy until the Buneman failure.
The project is at a good point:
A relatively simple sensor of two counterwound coils of relatively light wire (#24) on a simple pressure treated wood stand is fine.
Only one 125 ml bottle needs a sample (only about 4 Oz) of fluid. RainX or Prestone de-Icer windshield washer fluid works fine down to -25 F to -35 F.
A hybrid FET - relay switching circuit powers only one of the two sensor coils. The small signal relay "sets up" the connections, and the FET switches the realtively low polarization current (1 to 1.5 A). The relay only switches at zero coil current.
A front end amplifier with a common mode input filter and an ADI SSM2019 amplifier followed by a second order active bandpass filter provides adaquate signal to another passive common mode filter in the lab. A final gain of 10 in the lab is temporarily provided by PARC 113 preamp.
Both 12 bit digitization (NI USB 6008 at 5 or 6 kHz) and a Keithley 194A 16 bit digitizer provide good results.
Post analog filtering using FIR bandpass filters with 4 to 512 taps have worked well, narrow band tracking filters have worked (dynamic shot to shot calculation of filter paramaters) as have several IIR filters.
What is most needed now is a reliable frequency estimator with 10 to 1 milli Hz resolution. Both Koehler and Hollo have published frequency estimatation routines which might be used. Also, a brief search shows other possible DSP approaches. For example, MIT Professor Johnson's harminv code, based on a new FDM solution technique (1997) by Professor Mandelshtam at UC Irvine, looks very interesting.
It is still early in the project, and it is still unclear if the project should use a package such as LabView, using application builder for distribution, or Matlab, or if it should have custom software.
As an aside, I did some more work using an externally triggered hp counter to make a simultaneous hardware count of the field. While it is producing reasonable 100 point averages, the scatter shot to shot is several nanoTesla, much higher than the previous Buneman work. The Buneman estimator gaps are exceedingly frustrating, since otherwise the Buneman vi does a remarkable job.
Thursday, January 21, 2010
Just a quick update, with competing interests and several other part time endevors (which so far, better pay the bills!), work has been sporatic over the last couple of weeks.
Initial testing with harminv is very promising. Data sets were ported over to a LINUX machine (from LabView running on a WinXP machine) for testing. One interesting aspect of harminv is that it also provides amplitude and decay constant information. I am presently exploring ways to run harminv in the Windows environment, and hope to be able to call it from LabView, at least for continued R&D efforts.
I found harminv with a GoogleTM search while looking for a replacement for the Buneman frequency estimator around the first week of January, 2010, and learned about FDM at the MIT harminv page. (Afternote 5/24/2010, searching a key phrase "magnetometer harminv", I found a June 6, 2004 post (scroll up at the link to 8:06 am) by a Steve B. at the geotech magnetometer forum suggesting harminv based on the MIT page as a possibility for a pc based ppm. I had been reading posts around that time (as well as posting under the screen name Teneflin later in that thread), however I do not remember Steve's suggestion that someone try harminv. So, while this project might be the first FDM ppm or at least the first amateur FDM ppm, credit goes to Steve B. for first raising the possibility back in 2004. Back then, we were discussing 0.1 to 0.01 Hz resolution. FDM resolves to better than 0.001 Hz. While believed to be new to a geomagnetic observatory proton magnetometer application (at least new for an amateur observatory), FDM has been widely used in other NMR applications, especially in NMR spectroscopy applications.)
After much agonizing over how to try FDM in this magnetometer application, I found to date that harminv is NOT very portable to other platforms without a huge effort in the details of compiling and linking the various components. If this sort of thing is of interest, running a packaged harminv command line version (there are many for all flavors of LINUX) is worth the time. From there, move to other non-LINUX platforms at your own risk! The conversion has been done, generally as part of higher level physics programs where harminv is a component part, with programs such as cygwin and MinGW.
On further reading, I have been learning about NMR spectroscopy and Earth's field NMR ("EFNMR") spectroscopy. I will eventually include references for NMR spectroscopy on on a separate web page. At first look, it appears that spectroscopy experiments might require procedures and sample preparation beyond the scope of this experiment (still reading though!), however, there are hints of other measurement techniques that might be applicable here, for example digitizing the free induction decay signal (FID) (the precession decay signal) in quadrature in the time domain (from High Resolution NMR texts by Becker 2000, and Stejskal and Memory from 1994). More to follow.
Saturday, January 23, 2010
Last evening I re-read Professor Mandelshtam's paper, Vladimir A. Mandelshtam, Howard S. Taylor, Harmonic inversion of time signals and its applications, Journal of Chemical Physics (1997), Volume 107, Issue 17, 1997, Pages 6756-6769 describing his new FDM computational approach. Then I went back to work with his fortran code which he very generously provided and I was able to understand the FDM technique well enough to begin to apply the code in this magnetometer application.
Finally things started to make sense to me, and I am back on track after many distractions. It will take more time to adapt the code and parameters to this application, however as of this evening, FDM is up and running in LabView!!! (for now, as a system callable Windows executable, not yet as a dll or vi, but it seems to run fine this way in my LabView loops) If all continues to go well, tonight might be the first overnight run on FDM. This first run will be with the Keithley 194A high speed 16 bit digitizer, probably at 10 kHz with 10,000 points (FDM does not need a 2toN number of input points). This is very exciting, the first real move forward in some weeks!
As of now, my plan is to eventually make public all of the code (if the needed permissions are obtained) and circuit diagrams, but probably not until publication of the first project articles. Since much of what we are doing (we being GELLER (Geller Labs)) is not well tested and full of all sorts of errors here and there, there will likely not be distribution of code and/or schematics beyond the general design approaches and progress reports discussed here, for some time, probably not until project publication.
First couple of hours on FDM, with very little optimization: PDF. Looking good!!! Each point on the graph represents 10,000 data points at 10 kHz, ~24 seconds between points, 1.5A polarziation current, and still with the hybrid FET/small signal relay switching and the ADI SSM2019 based preamp out near the sensor presently running on Prestone windshield De-icer fluid. Th 24 seconds includes significant timed dead time. Probably the experiment will use 20 seconds per measurement to match the USGS methodology for the recording the scalar total F field.
I need to back the car out to get a first look at a 10 to 20 nT swing. It will be interesting to see if the reduction in gradient at the sensor (caused by the proximity of the car) improves the FDM figure of merit when the car is backed out to the road. I am of course hoping not to see another dead zone that was so problematic with the course nature of FFT based frequency estimators, such as the Buneman estimator that caused so much grief at 10 to 100 milli Hz resolution. The only way to increase the underlying FFT resolution on which estimators like Buneman are based, is to lengthen the digitization window. Lengthening the window, however, is not an option with the real world tau 2 of the exponentially decaying proton precession signal. On the other hand, a method such as Professor Mandelshtam's implementation of the FDM is ideally well suited to exactly this problem. See also: FDM: the Filter Diagonalization Method for data processing in NMR experiments, V.A. Mandelshtam, Progress in NMR Spectroscopy 2001, 38, pages 159-196.
For most experimenters, the FDM Harmonic Inversion routine will be a "black box" software processing module, do not be concerned about the advanced mathematics in the papers.
I would be remiss if I did not mention what a joy it has been to do this R&D work in LabView 2009. Also, I have more than used my software support payment this year, the National Instrument tech support has been excellent. I have called many times during this project and much of my progress is due the superb technical support that I received!
Sunday, January 24, 2010
The overnight run using the FDM to determine the precession frequency went well. Here is the overnight PDF, USGS 24 hour PDF. The figure of merit numbers go down as low as about 5E-7, however it is not clear yet how this computed parameter relates to actual error. FDM resolution is virtually unlimited, however the physical system (e.g. accuracy, jitter, and drift of the digitizing parameters including sample rate and sample window) and prevailing noise will ultimately determine the accuracy of the magnetometer. Our understanding is that the USGS scalar F data is a moving average of three measurements taken at 20 second intervals. We have held off on moving averages for now, since it is easier to determine the effect of passing vehicles by observing the raw individual measurement marks.
In Chemical analysis by ultrahigh-resolution nuclear magnetic resonance in the Earth's magnetic field, Nature Physics 2, 105 - 109 (2006) Stephan Appelt describes the ability to do NMR spectroscopy in the Earth's field (without the traditional big NMR magnet), especially where he stays >100 meters away from steel structures, achieving a deltaB/B of < 10-6 (page 106).
I just completed the first car move from ~30 feet to ~100 feet, from the sensor. PDF, TXT, USGS PDF. The first goal was to check for any Buneman type dead zones. Not present, FDM looks good! (notice the ambient field has a slight positive dB/dt during testing).
The next test was to leave the car out at the end of the driveway (not quite as far into the road as in the intial ramp test), ~delta 17 nT, to check for effects on the figure of merit. While the figure of merit varies measurement to measurement with the car in the garage, from roughly .0005 to 5e-7, and with a few failed measruements here and there (0 B field in the txt file) with the car out near the road, many more measurements have a figure of merit of < 5e-6, with many in the e-7 range, and some into the e-8 range. At first look, the scatter appears to be lower on the graph. I need to add the FDM amplitude and figure of merit to my txt file. This will need a more quantitative study.
Looks like I need to relocated the test stand, perhaps a project for the first warm days of spring. The sensor will be closer to passing cars, but on average it needs to be farther away from the parked car. (after notes: 1. During an unusually warm day of 40 F, I did try one alternative location, however, perhaps now too close to the overhead power lines, or possibly becasuse of earth return currents at that location, power line harmonics swamped the measurement. Tuning the FDM parameters might have solved the problem, however with another cold front approaching, I returned the sensor to its original location. 2. Since the car is mostly in the same place, most of the time, I simply added 18 nT to compensate for the car.)
Monday, January 25, 2010
Here is the overnight run using FDM and the Keithley 194A 16 bit digitizer (100 us, 8,000 samples, 20 seconds per measurement). PDF, TXT, USGS PDF.
It is easy now to see the influence of large ferrous masses (typically steel or steel alloys in vehicles) at a distance (not that surprising, since an early application of Earth's field magnetometry was anti-submarine warfare). This expanded view from about 6:30 am to 8:15 am shows garbage trucks (recyclables and waste) working their way down the road. PDF
For this temporary R&D configuration, the PAR 113 preamp gain in the lab was increased from 10 to 50 to make better use of the Keithley 194A +/- 3.2 V scale.
Tuesday, January 26, 2010
Here is the overnight run using FDM and the National Instruments NI USB 6008 12 bit digitizer (100 us, 8,000 samples, 20 seconds per measurement). PDF, TXT, USGS PDF. It is so quiet without the K194A fan running.
Forgot to mention that I added a +18 nT offset, since the car is usually in the substantially the same spot most of the time. Also, in preparing for FDM calibration runs, I noticed that while I was correctly sending corrected sample frequency to the digital filter, I had accidently truncated the sample interval being sent to FDM, so the FDM algorithm was running on an uncorrected input.
Performed calibration testing with FDM. The FDM algorithm solves correctly only for an exponentially decaying waveform, so now in addition to the GPS locked synthesizer sinewave an exponential window was applied to the test signal in Labview. The purpose of this testing is to correct for the actual digitizer sample rate (as compared to a GPS sync'd signal) as well as to look at linearity. I have not yet run an automated cycling at some interval, such as 10 milliHz, however random testing showed accuracy (after sample rate correction) to 1 milliHz with figure of merit numbers running e-8 to e-10.
FDM appears to be perfectly suited for use in magnetometer applications!
Wednesday, January 27, 2010
The FDM error, or figure of merit number, raises some interesting filtering possibilities. The run over night did not plot any measurements with a FOM of greater than 5e-6.
The normal measurement rate was changed to one measurement per minute. If a measurement failed (e.g. FOM > 5e-6), the system was programmed to keep trying at about 6 seconds per measurement until the FOM was acceptable.
Here are the results from the overnight run: PDF, TXT, USGS PDF.
At first look, it appears that some of the stray values, perhaps such as some of those caused by passing vehicles, have been removed.
There is still some measurable variation at the < 1 nT level measurment to measurement. It is unclear if that is measurement error, or if it is simply the actual variation of the Earth's field for this one minute measurement interval. My understanding is that the USGS raw F scalar is computed as the running average (boxcar) of the past 3 20 second values, so they might be intentionally filtering out such minute to minute variation as a matter of measurement convention: Temporal resolution 1-minute, 0.1667 Hz, Filter type, Boxcar, using 3 20-second samples (USGS, See: Proton Data (F) Specifications).
Here is an expanded view of the overnight data with a 1 nT (1 gamma) vertical resolution for some morning hours PDF. There is significant variation shot to shot at the <1nT level. Is it the power line, other man made magnetic disturbance, measurement error, gradient across the sensor (e.g. gradient caused by the nearby car at 33 feet) or actual fluctuation of the Earth's field?
Thursday, January 28, 2010
Overnight run PDF, TXT, USGS PDF.
I just started reading, High-Resolution NMR Techniques in Organic Chemistry, T.D.W. Claridge, Elsevier Science; 2nd edition, Nov, 2008. I noticed (page 43, zero filling and truncation artifacts) that applying an exponential window to avoid the discontinuity to zero at the end of the FID signal acquisition time window ( apodization ) might be helpful. As described above, to be able to calibrate using a CW GPS locked synthesized sinewave, an exponential window was applied to create an expontially decaying signal. Now the exponential window becomes an integral part of the signal processing routing (which incidentally should make calibration much easier).
With apodization, I was able to lower the FOM threshold from < 5e-6 to < 5e-7. Initial results look good, here is the evening PDF. The new routine seems to reject most moving vehicles and the moving snow plow, however if the plow pauses, such as to back up to clear an intersection, the data registers (the low points on the evening plot).
I am anxious to experiment with quadrature digitization techniques.
It is very windy today, gusts over 45 mph. There is some vibration in the relatively light test stand, not sure if mechanical vibration of the sample fluid can induce errors or affect the FDM FOM?
Friday, January 29, 2010
Battery pack for the front-end amplifier at the sensor test stand ran out around 4 am. Replaced batteries, now running at 1.8 seconds polarization pulse, 1.4 Amps, followed by single channel digitization at 6 kHz, 6,000 samples (1 second acquision window), exponential window factor 0.004. Measurements once per minute, with autocycling at about once per 6 seconds until the FOM is < 2e-6.
I had a terrible sinking feeling this evening when a gap showed up in the data PDF. First reaction was to zoom in - in time to look for any of the dithering about the non-linear regions and gaps that were present with the Buneman frequency estimator, but there are none, it is a clean shift PDF. Then I started looking out from the sensor (by the windows, it's -3F, a little chilly tonight), and I noticed several visiting vehicles parked out on the road at about 100 to 200 feet from the sensor! They are likely what caused the shift, all seems well. The Earth's field has been relatively noise this afternoon and evening, USGS PDF.
Saturday, January 30, 2010
Overnight run looks good: PDF, TXT, USGS PDF. The text file shows a number of failed measurements. It was -10 degrees F (-23c) early this morning and the batteries at the sensor are old recycled UPS batteries. That was probably the problem. The depressed section of the graph, as noted above, is from nearby vehicles parked on the road for a time.
It appears that the digital filter is no longer needed with FDM. Will run a few hours this pm without the digital filter: PDF.
Sunday, January 31, 2010
Overnight run: PDF,TXT, USGS PDF. Now that many of the passing vehicle disturbances have been removed by the FOM filter, I connected the cross marks with a graph line. There is still no moving or boxcar averaging and newly parked or departed vehicles still shift the curve, so the graph is a little rough. Otherwise all is running well on the NI USB 6008 module.
I might go back to the two channel Keithley 194A for quadrature digitization experiments. Not sure when those will begin. FDM will need to be reconfigured (just a switch setting) for imaginary number inputs, two columns, one for the in phase channel (I) and one for the quadrature channel (Q). The quadrature channel will be delayed from the I channel by 90 degrees at about 2291 Hz (our local fundamental precession peak) or by about 109 uS.
Project Documentation (very early stages)
QUESTIONS/COMMENTS/notice of typos, etc. send email to joegeller @ gellerlabs dot com
COPYRIGHT © 2009 JOSEPH M. GELLER, All rights reserved.