.. _intro_eval: ------------------------------------------ Evaluating the Results of Daily Processing ------------------------------------------ There are three primary criteria you should use to determine if your phase processing produced a good result: * All of the expected data were included. * The data fit the model at the expected level. * The uncertainties are acceptably small. In most cases you can assess these three requirements quickly using the GAMIT summary emailed to you by :program:`sh_gamit` (and saved as :file:`sh_gamit_{}.summary` in the day directory) together with the plots of daily repeatabilities created by :program:`sh_glred`. Following is the summary file from day 034 of the Southern California example: .. code-block:: text Input options -expt scal -d 2000 034 -netext a -orbit IGSF -noftp -remakex Y Processing 2000 034 GPS week 1047 4 Raw 0 /chandler/data30/rwk/active/example/034a Disk Usage: 37537 Free 31764.5 Mbyte. Used 55% Number of stations used 5 Total xfiles 5 Postfit RMS rms, total and by satellite RMS IT Site All 01 02 03 04 05 06 07 08 09 10 11 13 14 15 16 17 18 19 21 22 23 24 25 26 27 29 30 31 RMS 26 ALL 4.4 46 35 56 42 38 42 42 38 51 44 61 41 0 42 44 40 40 42 36 52 40 40 38 49 38 37 45 55 Best and Worst two sites: RMS 26 JPLM 3.2 3 3 4 3 3 3 3 3 4 3 3 3 0 3 3 3 3 3 3 4 3 3 3 4 3 3 4 4 RMS 26 MATH 3.5 4 2 5 4 4 3 4 4 4 4 4 3 0 3 3 3 3 3 3 4 3 3 3 4 3 3 3 4 RMS 26 7001 5.2 6 4 7 5 4 5 4 5 6 6 6 5 0 5 6 5 5 5 4 7 4 4 4 6 4 4 5 5 RMS 26 BLYT 5.5 6 4 7 5 5 5 6 5 6 5 7 5 0 6 6 5 5 5 4 6 5 5 4 6 4 5 5 7 Double difference statistics Prefit nrms: 0.25309E+01 Postfit nrms: 0.19379E+00 Prefit nrms: 0.25241E+01 Postfit nrms: 0.20241E+00 Prefit nrms: 0.25309E+01 Postfit nrms: 0.18607E+00 Prefit nrms: 0.25241E+01 Postfit nrms: 0.18889E+00 Number of double differences: 15098 Phase ambiguities (Total WL-fixed NL-fixed: 100 96 88 Phase ambiguities WL fixed 96.0% NL fixed 88.0% Check first that all of the data you expected got into the processing. The :content:`Total xfiles` should equal the number of raw or RINEX files available. If the :content:`Number of stations used` is less than the total x-files, then some x-files were created but then discarded because they were too small (tracking session too short or data discarded by :program:`makex`) as measured against the :content:`minxf` value you set in :file:`process.defaults`, or because you have excluded them with the :content:`xsite` option in :file:`sites.defaults` or the :program:`sh_gamit` command line. There are three indications of data quality and they each measure slightly different things. The block of lines beginning with :content:`RMS` are all extracted from :file:`autcln.post.sum` and show the one-way root-mean-square residuals by satellite and station. In the :content:`RMS` second line, the first value is an overall rms in mm. The remaining values on that line break down the scatter by satellite and report the value in tenths of mm to allow better discernment of differences that might indicate a bad orbital model for one satellite. The last four lines report the values for the two sites with the lowest (JPLM and MATH in the example) and the highest (7001 and BLYT) scatter. In a typical project, the best sites will have values of 3–5 mm, and the worst 7–9 mm. Values between 10 mm and 15 mm indicate high but acceptable levels of noise, but if you reweight the data using :program:`autcln` (default and recommended) the uncertainties in position estimates for these sites will be 2–3 times higher than for the better sites. RMS values greater than 15 mm suggest a poorly tracking receiver, a high multipath environment, severe weather, or a problem with convergence in :program:`autcln`, usually caused by poor coordinates or a short span of data. If the site's coordinates had large adjustments from the preliminary run (e.g., because this was the first day of processing a new site) and the RMS value is high, try simply rerunning the day with the now-updated coordinates. For sites with high scatter, examining the sky plots and phase versus elevation plots in the :file:`figs/` directory may help you understand the source of the noise. If both of the "worst" sites have high rms, you should look at :file:`autcln.post.sum` to check other sites, which might also have high values but did not make the "top two" in the summary. If either or both of the "best" two sites have 0 RMS, then all of the data from these sites were removed by :program:`autcln`. The most common reason was that :program:`autcln` inserted too many "bias" flags (detection of a possible cycle slip), either because of poor *a priori* coordinates or poor receiver performance. The tables in :file:`autcln.post.sum` allow you to determine the reason. If it is bad *a priori* coordinates, the pseudorange rms at the top of the file will make this immediately clear: .. code-block:: text AUTCLN SUMMARY FILE: Version 3.22I Clock and Range noise statistics at iteration 3 Site/PRN Allan SD@100 # Range rms # sec (ppb) (mm) 7001 1.552388 716 772.9 5353 ASH BLYT 0.448878 2876 329.3 21905 ASH JPLM 0.138763 2879 1101.2 19487 TRB LNCO 38.975374 2877 1026.4 19982 TRM MATH 100.000000 2876 482.8 18834 TRM Maximum values of 1–2 m, as in the example, are typical; if the value is greater than 5 m, a bad site coordinate is likely the problem. You can corroborate your assessment using the table reporting what :program:`autcln` did with bias flags: .. code-block:: text BIAS FLAG REPORT: Types ORG-Original JMP-Big Jump ION-Ion Jump GAP-Data Gap DDS-DD scan WLS-Wide Lane DDC-DD cleaning SITE # Flagged | # Remaining | # Edited | ORG JMP ION GAP DDS WLS DDC| ORG JMP ION GAP DDS WLS DDC| ORG JMP ION GAP DDS WLS DDC| 7001 0 0 17 21 1 0 0| 0 0 12 9 1 0 0| 0 0 5 12 0 0 0| BLYT 0 0 3 27 3 0 0| 0 0 0 6 1 0 0| 0 0 3 16 2 0 0| JPLM 0 0 32 266 16 0 0| 0 0 22 10 0 0 0| 0 0 10 222 11 0 0| LNCO 0 1 26 61 2 0 0| 0 0 8 9 0 0 0| 0 1 18 28 2 0 0| MATH 0 0 15 87 0 0 0| 0 0 14 10 0 0 0| 0 0 1 12 0 0 0| The block labeled :content:`# Flagged` reports how many bias flags were initially added scan for each station and the reason why—either gap or jump in the data. For example, the :content:`DDS` column indicates the number added during the scanning of the double-difference residuals. If this number is high, :program:`autcln` detected many discontinuities, which can be due to noisy phase data but would also result if there is a large slope in the residuals due to poor coordinates or orbits. The :content:`# Remaining` block reports how many flags remained after :program:`autcln` has finished its editing. You would like all stations to look like BLYT, but this is rarely the case. The :content:`# Edited` block reports the number of bias flags removed by deleting the data affected by the bias flag (as opposed to removing the flag by reliably resolving the integer number of cycles between data segments). Confirmation of :program:`autcln`'s action is given in the editing report: .. code-block:: text EDITING REPORT AND SITE PARAMS SITE nCLN MnOUT SNR LSNR GF03 RCLK GF02 BEND BCLS DDSC PFED GFUN BDL2 NODD ELEV EDIT MMRG ELCL Good (deg) (deg) L1 L2 7001 10.0 10.0 0 0 0 0 0 0 74 40 0 17 0 0 12 7 0 0 0 5210 BLYT 10.0 10.0 0 0 0 0 0 104 239 26 0 2 0 0 1091 3512 0 0 0 20547 JPLM 10.0 10.0 0 0 0 0 0 9 59 677 0 26 0 0 677 8 0 0 0 18055 LNCO 10.0 10.0 0 0 0 0 0 0 274 404 0 5 0 0 430 2076 0 0 0 19148 MATH 10.0 10.0 0 0 0 0 0 0 14 44 0 8 0 0 0 0 0 0 0 18768 Note whether each site has the expected number of "good" data (the original sampling interval for 7001 in this example was 120 s, compared with 30 s for the other receivers, so the number of data is what we would expect). If a site shows many data deleted, the column headers tell you the reason; see the :program:`autcln` help file and Section 4.2 of the `GAMIT Reference Manual `_ for an explanation of each header. The last part of the :program:`sh_gamit` summary file is extracted from the :program:`solve` q-file. The first four lines list, respectively, the normalized rms (square root of :math:`\chi^2` per degree of freedom) for the constrained solution with ambiguities free, constrained solution with ambiguities resolved, loose solution with ambiguities free, and loose solution with ambiguities resolved. With the current weighting of phase observations, all of these should be ~ 0.2. If the constrained nrms values are significantly larger than the loose nrms, the data are good but the constraints you have imposed in the GAMIT solution, either on coordinates (:file:`sittbl.`) or orbits (:file:`sestbl.`), are too tight. If this is the case, try constraining only the most reliable site and look at the adjustments of the others in the q-file. The last line in the summary gives the percentage of wide-lane (WL) and narrow-lane (NL) ambiguities resolved. (Note: This form of the summary is for :content:`LC_AUTCLN`; with :content:`LC_HELP` there are only two numbers, the first being the total number of ambiguities estimated, WL + NL, corresponding to twice the first number in the :content:`LC_AUTCLN` summary, and the second being the number of NL ambiguities resolved.) With :content:`LC_AUTCLN` more than 90% of the WL ambiguities should normally be resolved for any size network unless you have short sessions, noisy pseudoranges, and/or you are not using a :file:`dcb.dat` table that includes estimated values for the epoch of your data. The percentage of NL ambiguities resolved depends on the session length, size and configuration of the network, quality of the orbits and *a priori* coordinates, and atmospheric conditions. If it is less than ~ 80%, there may be deficiencies you can address with a different analysis approach. The definitive check of your data and processing is provided by the time series generated by :program:`sh_glred` as PostScript files in :file:`gsoln/`. With 24-hr sessions and a robust stabilization, you should obtain uncertainties and repeatabilities at the level of 1–2 mm for horizontal coordinates and 3–5 mm for heights. With 8-hr sessions, the horizontal should be 2–4 mm and height 10–15 mm. If the uncertainties are high for some sites but not others, check the session length (RINEX or x-file), number of data deleted (editing summary of :file:`autcln.post.sum`), and data noise (RMS table and :content:`Elevation angle dependent RMS statistics` of :file:`autcln.post.sum` and/or :content:`A priori receiver measurement error model` in the q-file). If the uncertainties for all sites are high, then check the record of the :program:`glred` and :program:`glorg` run, e.g. :file:`globk_scal_00034.org`: .. code-block:: text +++++++++++++++++++++++++++++++++++++ + GLORG Version 5.11I + +++++++++++++++++++++++++++++++++++++ Stabilization with 20.0% constant, 80.0% site dependent weighting. Delete sites with 4.0-sigma condition. Height variance factor 10.00 Position, 10.00 Velocity For Position: Min dH sigma 0.0050 m; Min RMS 0.0030 m, Min dNE sigma 0.00050 m For Velocity: Min dH sigma 0.0050 m/yr; Min RMS 0.0030 m/yr, Min dNE sigma 0.00010 m/yr Sigma Ratio to allow use: Position 2.00 Velocity 2.00 ======================================================================================== Starting Position stabilization iteration 1 L0002031200_scal.glx For 5 sites in origin, min/max hgt sigma 451.3 452.8 mm; Median 452.2 mm, Tol 10.0 mm Position system stabilization results --------------------------------------- X Translation (m) 0.24048 +- 0.00264 Iter 1 L0002031200_scal.glx Y Translation (m) 0.29123 +- 0.00398 Iter 1 L0002031200_scal.glx Z Translation (m) -0.25412 +- 0.00330 Iter 1 L0002031200_scal.glx Condition Sigmas used 0.0000 0.0000 0.0000 Sites and relative sigmas used in stabilization BLYT_GPS 1.00 7001_GPS 1.00 MATH_GPS 1.00 JPLM_GPS 1.00 LNCO_GPS 1.00 For 15 Position Iter 1 Pre RMS 0.0908 m; Post RMS 0.00362 m L0002031200_scal.glx For 5 sites in origin, min/max NE sigma 1.29 1.96 mm; Median 1.43 mm, Tol 1.00 mm The top part of the file records the sites included in the stabilization. As :program:`glorg` iterates the stabilization, it will remove sites that have large height uncertainties or horizontal and/or height residuals compared with the uncertainties. In the final iteration, there should be at least three (and preferable many more) sites left in the stabilization, and the :content:`Post RMS` should be at the level you expect for the uncertainties (1–5 mm). The next part of the file (not shown) echoes the :program:`globk` and :program:`glorg` command files (provided :content:`CMDS` was set in :content:`org_opt`); make sure these match what you intended. The next useful section is the position summary, which shows the adjustments and uncertainties of each of the sites in north, east, and up: .. code-block:: text SUMMARY POSITION ESTIMATES FROM GLOBK Ver 5.11I Long. Lat. dE adj. dN adj. dE +- dN +- RHO dH adj. dH +- SITE (deg) (deg) (mm) (mm) (mm) (mm) (mm) (mm) 245.285 33.610 -3.78 2.72 1.59 1.37 -0.062 37.86 6.07 BLYT_GPS* 243.531 34.560 0.12 -0.37 1.02 0.98 0.039 -9.53 4.71 7001_GPS* 242.563 33.857 1.33 -0.64 0.86 0.83 0.012 -1.86 4.04 MATH_GPS* 241.827 34.205 0.50 -0.51 0.85 0.80 0.031 -4.66 3.87 JPLM_GPS* 240.942 36.360 -0.27 0.42 1.43 1.37 -0.103 -8.82 5.64 LNCO_GPS* POS STATISTICS: For 5 RefSites WRMS ENU 1.36 1.01 14.27 mm NRMS ENU 1.30 1.02 3.07 The starred sites are those used in the stabilization (all in this case since the :file:`.apr` file is based on a previous solution); if there is sufficient redundancy, the individual adjustments will be useful in determining if the quality (rms) of the stabilization is degraded by the inclusion of particular sites. The stabilization statistics at the end of the position summary are shown for each component (north, east, up) rather than a combination of the three as given in the output at the top of the file. A more detailed discussion of :program:`globk` and :program:`glorg` output can be found in Section 3.5 of the `GLOBK Reference Manual `_.