This file includes the code to extract BUFR messages from NOAA bin files. More...
#include "bufrnoaa.h"
Include dependency graph for bufrnoaa.c:
Go to the source code of this file.
Functions | |
| int | main (int argc, char *argv[]) |
Variables | |
| int | STAGE |
| int | SELECT |
| int | INDIVIDUAL |
| int | COLECT |
| int | VERBOSE |
| int | LISTF |
| unsigned char | BUFR [BUFRLEN] |
| unsigned char | BUF [BLEN] |
| char | ENTRADA [256] |
| char | SEL [64] |
| char | SELS [64] |
| char | SELU [64] |
| char | SELO [64] |
| char | PREFIX [64] |
| char | HEADER_MARK |
| struct stat | INSTAT |
| char | OWN [] = "bufrnoaa" |
| char | SEP [] = "\r\r\n" |
| char | FINAL_SEP [4] |
Detailed Description
This file includes the code to extract BUFR messages from NOAA bin files.
*.bin files have the following sequence header: in the form '****NNNNNNN**** new line Name: This is taken as name \r\r
bufr report \r\r
First, it parses bufr reports from file with path included with argument -i Then for every report several decisions are taken depending on arguments 1) Select or discard depending on the type of BUFR To select we have to set our choice with the aid of argument -T. The argument supplied with '-t' will have one or more chars matching with T2 indexes (see below). As example, '-T USO' will select all the BUFR messages with T2 as U (upper level), S (surface) or O (oceanographic) If no '-t' argument is set it means NO SELECCTION i.e. we almost do nothing 2) For every selected BUFR message we can do any of these task, (they are not excluyent)
- To write individual files, one per bufr message, with names as 'YYYYMMDDHHmmss_ISIE06_SBBR_012100_RRB.bufr'. Date and time are source file timestamp and other items from bufr header. To do this we have to use '-f' option.
- To write an archive file as the original NOAA bin file, but just with selected messages. The option is '-F prefix' where prefix is the string to add. Resulting file names are in the form prefix_original_nane if no -F option then no archive bin file is generated. timestamp of resulting file is the same than the input file. In case of no bufr selected it just create a void file. Second item of resulting name file is 6 characters long: For observed data T1 is 'I' Possible values for T2 when T1 = 'I' O Oceanographic/limnographic (water properties) P Pictorial S Surface/sea level T Text (plain language information) U Upper air X Other data types Z Mixed data types When T1T2 = 'IS' values for A1: IS A 01–29 Routinely scheduled observations for n/a 000/006 distribution from automatic (fixed or mobile) land stations (e.g. 0000, 0100, … or 0220, 0240, 0300, …, or 0715, 0745, ... UTC) IS A 30–59 N-minute observations from automatic (fixed n/a 000/007 or mobile) land stations IS B Radar reports (parts A and B) RADOB 006/003 IS C 01–45 Climatic observations from land stations CLIMAT 000/020 IS C 46–59 Climatic observations from marine stations CLIMAT SHIP 001/0202009 edition ATTACHmENT II-5 I IS D Radiological observation RADREP 010/001 IS E Ozone measurement at surface n/a 008/000 IS F Source of atmospherics SFAZI, SFLOC, SFAZU 000/030 IS I 01–45 Intermediate synoptic observations from fixed SYNOP (SIxx) 000/001 land stations 000/051 IS I 46–59 Intermediate synoptic observations from mobile SYNOP mOBIL 000/004 land stations IS M 01–45 Main synoptic observations from fixed land SYNOP (SMxx) 000/002 stations 000/052 IS M 46–59 Main synoptic observations from mobile land SYNOP mOBIL 000/005 stations IS N 01–45 Synoptic observations from fixed land stations SYNOP (SNxx) 000/000 at non-standard time (i.e. 0100, 0200, 0400, 000/050 0500, ... UTC) IS N 46–59 Synoptic observations from mobile land stations SYNOP mOBIL 000/003 at non-standard time (i.e. 0100, 0200, 0400, 0500, ... UTC) IS R Hydrologic reports HYDRA 000/040 IS S 01–19 Synoptic observations from marine stations SHIP 001/000 IS S 20–39 One-hour observations from automatic marine n/a 001/006 stations IS S 40–59 N-minute observations from automatic marine n/a 001/007 stations IS T 01–19 Tide gauge observations n/a 001/030 IS T 20–39 Observed water level time series n/a 001/031 IS V Special aeronautical observations (SPECI) SPECI 000/011 IS W Aviation routine weather observations (mETAR) mETAR 000/010 IS X Other surface data IAC, IAC FLEET When T1T2 = 'IU' valores for A1: IU A Single level aircraft reports (automatic) AmDAR 004/000 IU A Single level aircraft reports (manual) AIREP/PIREP 004/001 IU B Single level balloon reports n/a IU C (used for single level satellite-derived SAREP/SATOB 005/000 reports – see Note 3) IU D Dropsonde/Dropwindsondes TEmP DROP 002/007 IU E Ozone vertical sounding n/a 008/001 IU I Dispersal and transport analysis n/a 009/000 IU J 01–19 Upper wind from fixed land stations (entire PILOT (parts A, 002/001 sounding) B, C, D) IU J 20–39 Upper wind from mobile land stations (entire PILOT mOBIL 002/003 sounding) (parts A, B, C, D) IU J 40–59 Upper wind from marine stations (entire PILOT SHIP 002/002 sounding) (parts A, B, C, D) IU K 01–19 Radio soundings from fixed land stations TEmP (parts A, B) 002/004 (up to 100 hPa) IU K 20–39 Radio soundings from mobile land stations TEmP mOBIL 002/006 (up to 100 hPa) (parts A, B) IU K 40–59 Radio soundings from marine stations TEmP SHIP (parts A, B) 002/005 (up to 100 hPa) IU M Model derived sondes IU N Rocketsondes IU O Profiles of aircraft observations in ascending/ AmDAR 002/020 descending IU P Profilers PILOT 002/010 IU Q RASS temperature profilers TEmP 002/011 IU R (used for radiance data – see Note 3) IU S 01–19 Radiosondes/pibal reports from fixed land TEmP (parts A, B, C, D) 002/004 stations (entire sounding) IU S 20–39 Radio soundings from mobile land stations TEmP mOBIL (parts A, 002/006 (entire sounding) B, C, D) IU S 40–59 Radio soundings from marine stations TEmP SHIP (parts A, 002/005 (entire sounding) B, C, D) IU T (used for satellite-derived sondes – see Note 3) SATEm, SARAD, SATOB IU U 01–45 Monthly statistics of data from upper-air stations CLImAT TEmP 002/025 IU U 46–59 Monthly statistics of data from marine stations CLImAT TEmP, SHIP 002/026 IU W 01–19 Upper wind from fixed land stations (up to PILOT (parts A, B) 002/001 100 hPa) IU W 20–39 Upper wind from mobile land stations (up to PILOT mOBIL 002/003 100 hPa) (parts A, B) IU W 40–59 Upper wind from marine stations (up to PILOT SHIP 002/002 100 hPa) (parts A, B) IU X Other upper-air reports for T1T2 = 'IO' values for A1: IO B Buoy observations BUOY 001/025 IO I Sea ice IO P Sub-surface profiling floats TESAC 031/004 IO R Sea surface observations TRACKOB 031/001 IO S Sea surface and below soundings BATHY, TESAC 031/005 IO T Sea surface temperature IO W Sea surface waves WAVEOB 031/002 IO X Other sea environmental About A2 A 0 - 90W northern hemisphere B 90W - 180 northern hemisphere C 180 - 90E northern hemisphere D 90E - 0 northern hemisphere E 0 - 90W tropical belt F 90W - 180 tropical belt G 180 - 90E tropical belt H 90E - 0 tropical belt I 0 - 90W southern hemisphere J 90W - 180 southern hemisphere K 180 - 90E southern hemisphere L 90E - 0 southern hemisphere M Lower left 10S 100E/upper right 70N 110W N Northern hemisphere P Area between 64.69N - 136.76W, 55.61N - 13.43W 64.69N - 156.76W, 55.61N - 33.43W S Southern hemisphere T 45W - 180 northern hemisphere U Area between 21.0N - 128.1W, 36.0N - 130.9W 21.1N - 113.0W, 36.2N - 110.5W V Area between 30.3N - 83.7W, 51.0N - 68.9W 19.8N - 64.5W, 33.3N - 47.1W X Global Area (area not definable)
Definition in file bufrnoaa.c.
Function Documentation
◆ main()
| int main | ( | int | argc, |
| char * | argv[] | ||
| ) |
Definition at line 210 of file bufrnoaa.c.
211{
212 size_t nb = 0, nc, nx = 0, nbuf = 0, nsel = 0, nerr = 0, i, nh = 0, nw;
213 FILE* ficin;
214 FILE* ficout = NULL;
215 FILE* ficol = NULL;
216 unsigned char b[4], header[256];
217 unsigned int expected = 0;
218 char name[256], namex[256], namec[256];
219 double tx;
220 struct timeval tini, tfin, tt;
221
222 // Initial time
223 gettimeofday ( &tini, NULL );
224
225 // read args from stdio
227 {
228 exit ( EXIT_FAILURE );
229 }
230
231 // Check input
233 {
235 exit ( EXIT_FAILURE );
236 }
237
238 // Open output file
240 {
242 exit ( EXIT_FAILURE );
243 }
244
246 {
247 // To make an archive
248
249 // build a name
250 strcpy ( namec, PREFIX );
251 strcat ( namec, ENTRADA );
252
253 // open the file
254 if ( ( ficol = fopen ( namec, "w" ) ) == NULL )
255 {
257 exit ( EXIT_FAILURE );
258 }
259 }
260
261
262 STAGE = 0;
263
264 memset ( &b, 0, 4 * sizeof ( unsigned char ) );
265
267 {
268 for ( i = 0; i < nc ; i++ )
269 {
270 // ingest a byte
271 b[3] = b[2];
272 b[2] = b[1];
273 b[1] = b[0];
274 b[0] = BUF[i];
275
277 {
278 case 0: // begining, searching header init
280 {
281 memset ( &header[0], 0, 64 * sizeof ( unsigned char ) );
282 header[0] = HEADER_MARK;
283 header[1] = HEADER_MARK;
284 header[2] = HEADER_MARK;
285 header[3] = HEADER_MARK;
286 nh = 4;
287 STAGE = 1;
288 }
289 break;
290 case 1: // header found, filling
291 if ( nh < 255 )
292 header[nh++] = b[0];
293 else
294 {
295 STAGE = 0;
296 nerr++;
297 break;
298 }
300 {
301 STAGE = 2;
302 //header[nh++] = '\0';
303 //printf("Header: '%s'\n",header);
304 }
305 break;
306 case 2: // header already finished
307 if ( nh < 255 )
308 header[nh++] = b[0];
309 else
310 {
311 STAGE = 0;
312 nerr++;
313 break;
314 }
315 if ( b[0] != 0x0a && b[0] != 0x0d )
316 {
317 STAGE = 3;
318 memset ( &name[0], 0, 128 * sizeof ( unsigned char ) );
319 name[0] = b[0];
320 nx = 1;
321 }
322 break;
323 case 3: // name already inited
324 if ( nh < 255 )
325 header[nh++] = b[0];
326 else
327 {
328 STAGE = 0;
329 nerr++;
330 break;
331 }
332 if ( nx >= 255 )
333 {
334 STAGE = 0;
335 break;
336 }
337 if ( b[0] == 0x01a || b[0] == 0x0d )
338 {
339 STAGE = 4;
340 name[nx++] = '\0';
341 //printf("Name: '%s'\n",name);
342 }
343 else if ( b[0] == ' ' )
344 name[nx++] = '_'; // substitute a space by '_'
345 else
346 name[nx++] = b[0];
347 break;
348 case 4: // Waiting BUFR message begin
349 if ( nh < 255 )
350 header[nh++] = b[0];
351 else
352 {
353 STAGE = 0;
354 nerr++;
355 break;
356 }
358 {
359 STAGE = 5;
360 expected = 0;
361 // printf("Leyendo BUFR\n");
363 nb = 4;
368 nh -= 4;
369 header[nh] = '\0';
370 }
371 break;
372 case 5: // Filling BUFR message till final '7777'
374 BUFR[nb++] = b[0];
375 else
376 {
378 fclose ( ficin );
379 exit ( EXIT_FAILURE );
380 }
381 // check expected length
382 if ( nb == 7 )
383 {
384 expected = ( unsigned int ) b[0] + ( unsigned int ) b[1] * 256 + ( unsigned int ) b[2] * 65536;
385 }
386
387 // Has been detected some void and fakes bufr
389 b[1] == HEADER_MARK &&
390 b[2] == HEADER_MARK
391 )
392 {
393 // Ooops. a fake bufr
394 // it seems a new header has been found before '7777'
395 STAGE = 0;
396 nerr++;
397 break;
398 }
399 if ( nb == expected )
400 {
402 {
403 STAGE = 0;
404 nbuf++;
406 printf ( "%s\n", name );
408 {
409 nsel++;
411 {
412 // prefix with input file timestamp
414 strcat ( namex,"_" );
415 strcat ( namex, name );
416 strcat ( namex, ".bufr" );
417 if ( ( ficout = fopen ( namex, "w" ) ) == NULL )
418 {
419 printf ( "Error: cannot open %s\n", name );
420 fclose ( ficin );
421 exit ( EXIT_FAILURE );
422 }
424 {
425 printf ( "Error: Writen %lu bytes instead of %lu in %s file\n", nw, nb, namex );
426 fclose ( ficin );
427 fclose ( ficout );
428 exit ( EXIT_FAILURE );
429 }
430 // close an individual fileq
431 fclose ( ficout );
432 // change individual file timestamp
434 }
436 {
437 // first write header
438 if ( ( nw = fwrite ( &header[0], sizeof ( char ), nh, ficol ) ) != nh )
439 {
441 fclose ( ficin );
442 fclose ( ficol );
443 exit ( EXIT_FAILURE );
444 }
445
446 // then bufr message
448 {
450 fclose ( ficin );
451 fclose ( ficout );
452 exit ( EXIT_FAILURE );
453 }
454 // finally \r\r\n
456 {
458 {
459 printf ( "%s: Error: Writen %lu bytes instead of 3 chars separing messages in %s\n", OWN, nw, namex );
460 fclose ( ficin );
461 fclose ( ficout );
462 exit ( EXIT_FAILURE );
463 }
464 }
465 }
466 }
467 }
468 else
469 {
470 // reached the expected end of BUFR without a '7777'
471 STAGE = 0;
472 nerr++;
473 break;
474 }
475 }
476 }
477 }
478 }
479
481 {
482 fclose ( ficol );
483 // change archive file timestamp
485 }
486
487 // Final time
488 gettimeofday ( &tfin, NULL );
489 fclose ( ficin );
490
491 // A brief stat output
493 {
494 printf ( "Found %lu bufr reports. Selected: %lu. Wrong: %lu\n", nbuf, nsel, nerr );
495 timeval_substract ( &tt, &tfin, &tini );
496 tx = ( double ) tt.tv_sec + ( double ) tt.tv_usec *1e-6;
497 printf ( "%lf seg. ", tx );
498 if ( nbuf && tx != 0.0 )
499 printf ( "%lf reports/sec.\n", ( double ) nbuf / tx );
500 else
501 printf ( "\n" );
502 }
503 exit ( EXIT_SUCCESS );
504}
int bufr_is_selected(char *name)
returns 1 if selected message 0 otherwise
Definition: bufrnoaa_utils.c:124
int is_head_custom(unsigned char *b, char mark)
checks if an unsigned char from an array is the first char of repeated mark char four times
Definition: bufrnoaa_utils.c:61
int is_endb(unsigned char *b)
checks if an unsigned char from an array is the first char of '7777'
Definition: bufrnoaa_utils.c:76
int mtime_from_stat(char *filename, struct stat *st)
modifies a file update time from a struct stat
Definition: bufrnoaa_utils.c:178
int date_mtime_from_stat(char *date, struct stat *st)
get a string with date and time from a struct stat
Definition: bufrnoaa_utils.c:163
int is_bufr(unsigned char *b)
checks if an unsigned char from an array is the first char of 'BUFR'
Definition: bufrnoaa_utils.c:33
int timeval_substract(struct timeval *result, struct timeval *x, struct timeval *y)
Subtract the ‘struct timeval’ values X and Y, storing the result in RESULT.
Definition: bufrnoaa_utils.c:93
References BLEN, BUF, BUFR, bufr_is_selected(), BUFRLEN, COLECT, date_mtime_from_stat(), ENTRADA, FINAL_SEP, HEADER_MARK, INDIVIDUAL, INSTAT, is_bufr(), is_endb(), is_head_custom(), LISTF, mtime_from_stat(), OWN, PREFIX, read_args(), STAGE, timeval_substract(), and VERBOSE.
Here is the call graph for this function:
Variable Documentation
◆ BUF
| unsigned char BUF[BLEN] |
Definition at line 195 of file bufrnoaa.c.
Referenced by main().
◆ BUFR
| unsigned char BUFR[BUFRLEN] |
Definition at line 194 of file bufrnoaa.c.
Referenced by main().
◆ COLECT
| int COLECT |
Definition at line 192 of file bufrnoaa.c.
Referenced by main(), and read_args().
◆ ENTRADA
| char ENTRADA[256] |
Definition at line 196 of file bufrnoaa.c.
Referenced by main().
◆ FINAL_SEP
| char FINAL_SEP[4] |
Definition at line 207 of file bufrnoaa.c.
Referenced by main(), and read_args().
◆ HEADER_MARK
| char HEADER_MARK |
Header mark character who is repeated four times at the begining
Definition at line 202 of file bufrnoaa.c.
Referenced by main(), and read_args().
◆ INDIVIDUAL
| int INDIVIDUAL |
Definition at line 192 of file bufrnoaa.c.
Referenced by main(), and read_args().
◆ INSTAT
| struct stat INSTAT |
Definition at line 204 of file bufrnoaa.c.
Referenced by main().
◆ LISTF
| int LISTF |
if != then a list of messages in bin file is generated
Definition at line 193 of file bufrnoaa.c.
Referenced by main(), and read_args().
◆ OWN
| char OWN[] = "bufrnoaa" |
Definition at line 205 of file bufrnoaa.c.
Referenced by main(), print_usage(), print_version(), and read_args().
◆ PREFIX
| char PREFIX[64] |
Definition at line 201 of file bufrnoaa.c.
Referenced by main(), and read_args().
◆ SEL
| char SEL[64] |
Selection string for argument -T according with T1
Definition at line 197 of file bufrnoaa.c.
◆ SELECT
| int SELECT |
Definition at line 192 of file bufrnoaa.c.
Referenced by read_args().
◆ SELO
| char SELO[64] |
Selection string for A1 when T2='O' (argument -O)
Definition at line 200 of file bufrnoaa.c.
Referenced by bufr_is_selected(), and read_args().
◆ SELS
| char SELS[64] |
Selection string for A1 when T2='S' (argument -S)
Definition at line 198 of file bufrnoaa.c.
Referenced by bufr_is_selected(), and read_args().
◆ SELU
| char SELU[64] |
Selection string for A1 when T2='U' (argument -U)
Definition at line 199 of file bufrnoaa.c.
Referenced by bufr_is_selected(), and read_args().
◆ SEP
| char SEP[] = "\r\r\n" |
Definition at line 206 of file bufrnoaa.c.
Referenced by read_args().
◆ STAGE
| int STAGE |
Definition at line 192 of file bufrnoaa.c.
Referenced by main().
◆ VERBOSE
| int VERBOSE |
Definition at line 192 of file bufrnoaa.c.
Referenced by bufrtotac_read_args(), main(), and read_args().
