-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathAN-1.5
3170 lines (2150 loc) · 114 KB
/
AN-1.5
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Star is the fastest tar archiver for UNIX
Star has many improvements compared to other tar
implementations (including gnu tar). See below for a short description
of the highlight of star.
Star is located on:
ftp://ftp.berlios.de/pub/star
Changes since star-1.4:
/*--------------------------------------------------------------------------*/
Between star-1.4 and star-1.5, there have been more changes than in the
whole life of star untill star-1.4.
Important changes as an overview:
- Changed license from GPL to CDDL
- There are really many new features and the codesize was doubled
- star now includes code to emulate different Command line interface types.
The following CLI are currently implemented:
- star -> the classical star CLI
- suntar -> the UNIX SUSv2 OpenGroup standard CLI with Sun
enhancements
- cpio -> the UNIX SUSv2 OpenGroup standard CLI with SVr4
enhancements
- pax -> the POSIX.1-2001 standard CLI
- gnutar -> the CLI from GNU tar 1.14
The command lind interface is automatically selected from argv[0].
- True and reliable incremental backup/restore has been implemented using
the basic ideas from ufsdump/ufsrestore. Whether a file gets archived
on an incremental depends on the dump level and the time of the last
incremental or full backup. Whether a file needs to be renamed or
removed while restoring incrementals depends on the data in a database
that is created for the period of time of the incremental restore. The
database contains file names as well as old and new inode numbers.
- A configurable error control mechanism allows to selectively ignore
certain problems.
- Star now implements a fast and reliable -copy option.
- Star now implements true and efficient Sparse file support for
Solaris 11 (and Solaris 10 updates).
- Star now implements reliable multi-volume support with automated tape
end/size recognition
- Star now uses "libfind" and thus allows to use find(1) syntax in star
command lines.
- New program "tartest" reads TAR archives from "stdin" and prints
a list of non-compliances with POSIX.1-1990
Call e.g.
tar cf - | tartest
To check your favorite TAR implementation against POSIX.1-1990
/*--------------------------------------------------------------------------*/
All changes in a fine grained list:
- Significant speed up for creating POSIX.1-2001 extended TAR headers
When star creates archives with POSIX.1-2001 extended TAR headers,
it needs to create twice as many TAR headers as with 'classic' TAR
archives. In addition the POSIX.1-2001 meta information needs to be
coded.
While it is impossible to speed up the creation of the additional
TAR headers, the coding of the POSIX.1-2001 meta information has now
been implemented in a way that now only spends half of the time
compared to the time used up by star-1.4
- Speed up the recognition of POSIX.1-2001 extended header tags
- Diff code now is able to compare the number of hard links to
a file if the archive is in -dump (exeustar) format.
- List code now includes hard link count in the long listing if the
archive is in -dump (exeustar) format.
This allows to see the difference in the number of hard links
in diff mode if the archive is in POSIX.1-2001 extended -dump
(exeustar) format.
- Diff code now is able to compare the directory content if the
archive is in -dump (exeustar) format.
This allows star to tell whether the target directory on disk
includes more files than the archive. This property is
needed in incremental restore mode to remove files that
have been removed between incremental dumps.
NEW FEATURE: you now can compare two directory trees on disk
with only one run of star by calling:
star -C fromdir -c -dump . | star -C todir -diff -v
or
star -C fromdir -c -dump . | star -C todir -diff -vv
Star will now even complain if the directory 'todir' contains
more files than the directory 'fromdir'.
- New diffopt= 'dir' to tell star whether to compare or
dir content if the archive is in -dump (exeustar) format.
- Enhanced autoconf code for OS/2
- Add socket lib definition to RULES to allow networking programs
to work
- Check whether readlink() is present to allow compilation on OS/2
- Check whether symlink() is present to allow compilation on OS/2
- Check whether chown() is present to allow compilation on OS/2
- fexec.c in libschily now uses ';' as PATH Environment delimiter
for EMX to allow path search to work on OS/2 too.
- Call setmode(stdin/stdout, O_BINARY) on OS/2 EMX too.
- Now call setmode(pipe, O_BINARY) on _both_ sides of the compress pipe.
This should affect OS/2 _and_ Cygwin.
- Work around a OS/2 EMX bug that causes star to dump core if the
FIFO code correctly closes the unused parts of the pipes that
are used o synchronize the cache.
This was caused by attaching the shared memory segment too late
to the child process.
- In create mode try always to avoid to need to call
telldir()/closedir()/opendir()/seekdir()
by reading the directory content into allocated memory.
This may cause problems on systems that are low with memory.
This is implemented in the new function fetchdir().
- New option -lowmem to tell star to behave as before.
With -lowmem is used, star does not malloc() the dir content
and defaults to 1 MB FIFO size (see above)
- include the directory content (names and inode numbers) in the
extended POSIX.1-2001 header if in -dump mode.
Star uses a vendor unique extended POSIX.1-2001 header tag named
SCHILY.dir
The tags
SCHILY.ddev
SCHILY.dino
are used with incremental dumps. They are used only if the dump level
is greater than 0. They are neded to allow star in incremental restore
mode to know which files have been removed before the tape has
completely passed. For this reason, star will be able
to do incremental restores even if the filesystem is nearly full.
- Fixed a bug introduced by applying a patch from 8.3.2002
from Andreas Gruenbacher <[email protected]>
The bug caused a needed ',' to be omited from the ACL string
in extract mode if the target platform did not have named entries
in the passwd file and the ACL string was converted to use the
numeric user id instead.
- Fixed a bug that caused star to complain
"star: Cannot handle multi volume archives from/to stdin/stdout."
if called "star -c f=- ..." or "star cf - ...".
- Fixed a bug that did not allow it to specify the keyword "nlink" with the
diffopts= option. Old star versions did not implement this option and thus
it has been forbidden before.
- Make sure that Base 256 coding is only used in the "classical tar headers"
if the receiving archive format will understand it.
- Fixed a bug in star that caused star to create wrong filenames if the
POSIX.1-1990 file name prefix was exactly 155 chars _and_ the
reserved field in the TAR header that directly follows the prefix
field included non-null characters.
- Call fflush(vpr) before executing volume change script
to sync output from star with output from the script.
Thanks to a hint from Abramo Bagnara <[email protected]>
- Do not open /dev/tty in case that a volume change script
has been specified. This allows star to be called from
a cron script.
- Fix a minor bug in the new diff code. The bug caused that
star -diff -vv used on TAR archives created with -dump
not to print all directory differences.
- If one of both (the dir in the archive or the dir on
disk) was empty no file list was printed
- If one of the sorted lists was longer than the other
starting from some point, the rest of filenames
was not printed.
The pure fact that the directories were different however
was always printed.
- New directory 'testscrips/' with data for the program 'tartest'.
- A 20 year old extension has been removed from libschily/format.c
This caused printf() to interpret %X as %lX. This caused noticable
problems on IA-64. It should have created problems on True64
and on 64 bit Sparc programs too but was not directly visible
due to a different stack content.
- #elif removed to avoid GCC-3.x warnings
- several new autoconf tests for 'struct mtget' members
- remote tape client code now uses a new abstract 'struct rmtget'
to avoid that 'mt status' for a remote tape drive depends on
the least common denominator from both (local and remote) systems.
- -wready option now also looks for EBUSY which allows
to use star and mt to wait for remote tape drive to become
ready again. This option should be used if you like to do a series
of backups as it may be that writing the EOF mark blocks
the tape for a while.
- -wready now waits up to 120 seconds for the tape drive to
become ready
- star manpage enhanced to reflect the new sense for -wready
- With FIFO enabled (default since 1994) star is now able to
exit() with a correct exit code != 0 if a write error occurs
on the tape. Due to a missing feature, star did exit(0) if the
fifo was active.
- star man page enhanced to mention the new diff properties that
are present when using the -dump option.
- Remote tape client code has been made a bit faster (5-10%)
- Remote tape client code now includes a new interface for an
abstract 'struct mtget'. This allows the mt program to work
more smoothly when issuing a remote status command.
Before the result was caused by the least common denominator
of both, the local and the remote system. With the enhancements,
the outout of the mt program with the 'status' opcode only depends
on the properties of the remote system where the tape is connected
to.
- mt program now also supports -wready option
- mt program has new opcode 'nop' to allow mt to be used
as stand alone program to wait for a tape to become ready
by calling "mt -wready nop"
- mt program now supports the new abstract remote tape interface
for struct mtget.
- Remote MT client code moved into 'librmt'.
I also modified the code in a way that allows to compile librmt
in a way that does not need libschily in order to work if the
OS is at least UNIX-98 compliant.
This makes it possible to use a complete portable remote tape access
solution in any application.
If someone is interested to get a BSD licensed version of librmt
please contact me!
- Special support for an ndocumented but unfortunately important
"feature" for symlinks on HP-UX.
Now star tries to retain the permissions of a symlink by
setting umask() before creating the symklink.
- Fixed a bug in _fileread() that caused a lseek() return code
not to be evaluated correctly.
- Fixed a bug in create_dirs() that may have caused incorrect behavior
when a non-directory should be converted into a directory.
- Fixed a bug in create_dirs() that caused an endless loop if trying
to extract e.g.
dir1/dir2/file
When
dir1
Exists and may not be written to.
- Auto-create dependencies for HP-UX bundled C-compiler
- Auto-create dependencies for AIX C-compiler
- Reworked 'librmt' now allows to use star in remote mode without
being installed suid root.
To force star to use rsh(1) instead of rcmd(3), use
env RSH=rsh star ...
To tell star to use a different RMT server, call
env RMT=/path/to/rmt star ....
NOTE: Transfer speed is much slower if star does not use rcmd(3)
in root mode to establish the remote connection. If you like to
do fast, always make sure to be root or install star suid root.
For late 2002, we know that the following programs
are broken and do not implement signal handling correctly:
rsh on SunOS-5.0...SunOS-5.9
ssh from ssh.com
ssh from openssh.org
Sun already did accept a bug report for 'rsh'. Meanwhile it could
help to call setsid() if we are running under X so the ssh
X pop up for passwd reading will work.
For Solaris the following Bug requests are open:
rsh/rlogin 4776756
sun ssh 4777436
The source from OpenSSH.org has been fixed on Monday 11.11.2002.
Due to the signal handling bug, star will die if you try to type
^\ (SIGQUIT) to get an intermediate status summary. Star will
also not be able to write the current file to a logical end before
stopping because it received a SIGINT - this will result in
archives with a defective logical structure.
- Reworked 'librmt' allows to use ssh(1) to be used for remote tape
access. Use
env RSH=ssh star ....
or
env RSH=/usr/bin/ssh star ....
to tell star to use the secure shell to establish a remote tape
connection. Note that using the secure shell causes the transfer
speed to be significantly lower than for the default case where
a suid root star used rcmd(3) to set up the connection.
- Man pages for librmt new.
- Include a patch from Andreas Gruenbacher to fix a problem with
illegal ACL comments in linux tar archives that caused star
to be unable to extract the archive.
- Add support for *BSD SIGINFO (SIGINFO has been added by BSD in 1990)
We do support intermediate status printing much longer but triggered
by SIGQUIT (^\).
- Major rework for "rmt" man page
- Make "rmt" return the string "PIPE" if "stdin" is connected to a pipe.
This allows to use ssh(1) for the remote tape interface.
- changes to reflect new interfaces of 'librmt'.
- 'mt' command renamed to 'smt' to allow it to be installed as
alternate mt program to be used for backup purposes
- follow the strange POSIX rules for the strerror() behavior
- Support for sparc64-freebsd
- Try to work around a bug in OpenBSD.
OpenBSD defines EOF in ctype.h but POSIX only mentions an EOF definition
for stdio.h. If we include ctype.h before schily.h on OpenBSD while
stdio.h has not been included, this will fail.
- Support for BSDi version 4.x
- Support for i786-cygwin32_nt
- RMT program now supports IPv6
- Fix a bug that caused star to dump core with star -diff -vv
if the archive has been created with star -c -dump and the
target directory in the filesystem was not readable.
- Fixed a bug that caused star to be unable to correctly deal with
ACLs where the user or group name contains spaces.
- Workaround for reading rotten archives caused by above bug.
- Fixed a bug that caused star to create broken archives if
the POSIX.1-2001 extended format is used for sparse files > 8 GB.
For this reason, a new extended header tag SCHILY.realsize has
been added.
- New option -no-xheader disables PORIX.1-2001 extended headers
regardless of the archiveformat. This may help to read archives
that have been created with previous versions of star if there
is a sparse file > 8 GB in the archive.
- New Option xdebug= #
Currently star xdebug=1 tells star tu print a dump of the internal
datatructures for every file ifin list or extract mode.
- Allow floating point printing on OS/2. This has been possible
since 11/2001 but OS/2 has been forgotten.
- Correct OS/2 ranlib handling
- New architecture 9000-831-hp
- Include +DAportable in HP-UX cc options
- Reordered and restructured librmt code to allow compilation on OS/2
and hopefully other OS like BeOS
- Fixed a bug that caused star to print '^Aname' instead of 'name'
in extended directory diff mode.
- Fixed a bug that caused star to dump core on UNIX-98 TAR compat mode
with e.g. 'star cbv' because star did not check for the missing arg to
the -b option.
- Fixed a bug in POSIX.1-2002 mode (SCHILY dir namelist extension) that
resulted in wrong directory name lists (used for diff mode only) if
the directory contained japanese filenames. The bug was present because
star did not move enough space if the filename length changed when
converting to Unicode.
- Fixed a typo (R_RDONLY instead of O_RDONLY) in fctldefs.h
- Support for macppc-netbsd-cc NetBSD on MAC HW.
- Fixed a minor typo in the star man page.
Thanks to Eric Raymond
- 'star -n -tpath ...' now only prints the pathnames to allow
to use the output directly in scripts. If you like the old
behavior, use 'star -nv ...'
- Better autoconfiguration for librmt and its users yields in better
compilation and portability results for OS that do not support the
needed internet features for remote tape support.
- mt.c adopted to the better autoconf support in librmt
- New option errctl= name
Use the file name as error control file. The reason
for using an error control file is to make star quiet
about error conditions that are known to be irrelevant
on the quality of the archive or restore run. A typical
reason to use error control is to suppress warnings
about growing log files while doing a backup on a life
file system.
The list of error conditions to be handled may use one
or more (in this case separated by a '|' character)
identifiers from the list below:
ABORT If this meta condition is included in an
error condition, star aborts (exits) as
soon as possible after this error condition
has been seen instead of making star quiet
about the condition. This error condition
flag may only be used together with at
another error condition or a list of error
conditions (separated by a '|' character).
WARN If this meta condition is included in an
error condition, star prints the warning
about the error condition but the error
condition does not affect the exit code of
star and the error statistics (which is
printed to the end) does not include the
related errors. This error condition flag
may only be used together with at another
error condition or a list of error condi-
tions (separated by a '|' character). The
WARN meta condition has a lower precedence
than ABORT.
DIFF Suppress output in case that star -diff did
encounter any differences.
ALL This is a shortcut for all error conditions
below.
STAT Suppress warnings that star could not
stat(2) a file.
GETACL Suppress warnings about files on which star
had problems to retrieve the ACL informa-
tion.
OPEN Suppress warnings about files that could
not be opened.
READ Suppress warnings read errors on files.
WRITE Suppress warnings write errors on files.
READLINK Suppress warnings readlink(2) errors on
symbolic links.
GROW Suppress warnings about files that did grow
while they have been archived.
SHRINK Suppress warnings about files that did
shrink while they have been archived.
MISSLINK Suppress warnings about files for which
star was unable to archive all hard links.
NAMETOOLONG Suppress warnings about files that could
not be archived because the name of the
file is too long for the archive format.
FILETOOBIG Suppress warnings about files that could
not be archived because the size of the
file is too big for the archive format.
SPECIALFILE Suppress warnings about files that could
not be archived because the file type is
not supported by the archive format.
GETXATTR Suppress warnings about files on that star
could not retrieve the extended file attri-
bute information.
SETTIME Suppress warnings about files on that star
could not set the time information during
extraction.
SETMODE Suppress warnings about files on that star
could not set the access modes during
extraction.
SECURITY Suppress warnings about files that have
been skipped on extraction because they
have been considered to be a security risk.
This currently applies to all files that
have a '/../' sequence inside when -.. has
not been specified.
LSECURITY Suppress warnings about links that have
been skipped on extraction because they
have been considered to be a security risk.
This currently applies to all link names
that start with '/' or have a '/../'
sequence inside when -secure-links has been
specified. In this case, star tries to
match the link name against the pattern in
the error control file.
SAMEFILE Suppress warnings about links that have
been skipped on extraction because source
and target of the link are pointing to the
same file. If star would not skip these
files, it would end up with removing the
file completely. In this case, star tries
to match the link name against the pattern
in the error control file.
BADACL Suppress warnings access control list
conversion problems.
SETACL Suppress warnings about files on that star
could not set the ACL information during
extraction.
SETXATTR Suppress warnings about files on that star
could not set the extended file attribute
information during extraction.
If a specific error condition is ignored, then the error
condition is not only handled in a silent way but also
excluded from the error statistics that are printed at the
end of the star run.
Be very careful when using error control as you may ignore
any error condition. If you ignore the wrong error condi-
tions, you may not be able to see real problems anymore.
A typical error control file line might look this way:
GROW */log/#[^/].log
- Typo in DEFAULTS files fixed
- Better Next STep support:
- Some workarounds for broken unistd.h
- -lkvm removed from Next Step config.
- waitdefs.h fixed for very old BSD based systems (~ 1980)
like Next STep
- fixed typo in fctldefs.h R_RDONLY -> O_RDONLY
- Better Portability for SCO UnixWare
- New platforms i486, i586, Pentium III, Pentium Pro
- Let strdefs.h also include strings.h for strcasecmp()
- Support for missing struct sockaddr_storage
- Support for broken wait3() (returns wrong timings)
- Changed broken portability Prototype support in libfile
to use 'makefiles / prototype.h' based system
- Changed fileopen() and filereopen() to avoid fdopen()
problems (does not accept mode string that does not
match fd flags)
- Change autoconf/statdefs.h to support nanosecond support for SCO
UnixWare and FreeBSD
- Call conf/mkdep-sco.sh via sh to be independent from 'x' bit.
- starting to port ACL support to SCO UnixWare (not yet ready)
- Removed option -I in preparation for using it in a different way
in future releases.
- Trying to make the source get accepted by 'cstyle'.
- changed conf/cc-config.sh to check for a bad PATH set up
(e.g. /usr/ucb before /opt/SUNWspro/bin) which would cause
problems to use cc on Solaris.
- Second fix from Andreas Grünbacher for #default comments
in ACLs now also works correctly with multiple #default comments.
- Warning: Environment STAR_FIFO_SIZE changed to STAR_FIFOSIZE
This is to make star and cdrecord behave similar.
- Star now reads a file /etc/default/star for default parameters.
Currently implemented are:
STAR_FIFOSIZE=
and
STAR_FIFOSIZE_MAX=
STAR_FIFOSIZE= sets the FIFO size if no fs= option has been specified
STAR_FIFOSIZE_MAX= limits the FIFO size even if a fs= option has been specified
- Try to avoid that the verbose or diagnostic messages are
sometimes lost if called on Linux via "ssh". Unfortunately,
this does not always help. If you like to make sure that
nothing gets lost, call: ssh host "star .... ; sleep 10"
NOTE: The basic problem is caused ba a Linux bug
- Star now opens a file that is going to be extracted with O_DSYNC.
Thanks to a hint from Frank Batschulat from the Solaris filesystem
kernel team.
On operating systems with a decent buffer cache implementation
(like Solaris), this speeds up extraction of many small or medium
sizes files (e.g. a Linux kernel TAR archive) by approx. 20%
On Linux, this change unfortunately is ineffective.
- New option -no-fsync to avoid calling fsync(2) for each file
if in extract mode.
This helps to increase extraction speed on operating systems
with badly implemented buffer cache (e.g. Linux).
Note that using this option may be dangerous as star will not
even be able to detect if a file could not be extracted correctly
if it has been called with -no-fsync
Background information:
Calling fsync(2) in star's extract mode has been introduced in 1999.
As star is developed on Solaris and on Solaris the performance degradation
that is caused by calling fsync(2) is close to imperceptible, no
performance tests have been made for this feature until recently.
A few weeks ago, some people reported that GNU tar is much faster
than star when extracting files.
Doing performance tests by extracting linux kernel TAR archives
leads to the following results:
- Extraction on Solaris without calling fsync(2) did speed up
by ~ 40% compared to the standard case with calling fsync(2).
- After O_DSYNC has been introduced, the speed difference
(compared to extraction without fsync) reduced to ~ 20%
Extraction of a recent 2.4 Linux developer kernel archive
took 50 seconds with star's default that calls fsync and
42 seconds with star -no-fsync (56 seconds with star-1.5a14)
- On Linux (depending on filesystem type and disk type)
extraction without calling fsync(2) is 3x to 8x faster
than with the standard case that includes calling fsync.
Star is developed for best possible data integrity while GNU tar
does not seem to care about data integrity. The fact that GNU tar
on Linux is much faster (for extraction) than star is no reason
to blame star but a reason to blame the quality of the Linux
buffer cache. Star will not change its default to a method that
gives less data integrity just to speed up compared to GNU tar.
The Solaris kernel is developed for best speed with guaranteed
data integrity while the Linux kernel is developed for best speed
without caring for data integrity. If you like that star will
speed up in the star default (secure) mode, send a message to the
Linux kernel developers team and request a change that gives a
better behaviour of the Linux buffer cache.
- New option level= # to set the current incremental dump level
- New option tardumps=name to set file name for tar dump dates
file. This file is similar to the file /etc/dumpdates that
is manages by ufsdump.
- New option -wtardumps to tell star update file for tar dump
dates if in dump mode.
If you like to experiment with the new incremental dump features,
you should start this way:
Create a level 0 dump:
star -cM -dump -sparse level=0 -wtardumps -C /usr .
Create a level 1 dump:
star -cM -dump -sparse level=1 -wtardumps -C /usr .
If you do not have files that changed the type, no files that
have been removed and no files that have been renamed,
star would already be able to do incremental restores.
- Better handling for the exitcode for star (needed for dump
handling).
- Allow 'smt' to understand that EACCES for read-only open may
not be caused by a write protected tape but rather by access
problems for a remote tape drive.
- Do no more archive the list of files in a directory
(usually done with -dump) if this directory is a mount
point and star has been told not to cross mount points.
We cannot use the real content of the underlying dir
anyway and the content of the mounted FS makes no sense.
Conclusion: try to use a filesystem snapshot whenever
possible.
- Fixed a problem when extracting extended file flags on FreeBSD
Thanks to Marius Strobl <[email protected]>
- Added a workaround for permission problems when trying to extract
hard links to immutable files by resetting the flags, creating the link
and later restoring the flags.
Thanks to Marius Strobl <[email protected]>
- made handlecond()/raisecond() portable in libschily
- Extended README.hpux to mention how to compile in 64 bit mode.
- Added a security check for "/../" in the path for the "rmt"
program that helps to prevent user from opening /dev/../etc/passwd
in case there is no /etc/default/rmt
- Add a security check in star's extract mode.
Files containing /../ will be skipped by default.
Exceptions are -w or -.. (the latter is a new option) has been specified
- New option -.. to tell star that extracting files containing /../ in
the path name is OK.
- New option -secure-links to tell star to skip hard links and symbolic links
that have a link name starting with / or contain "/../" in the link name.
Note that it is impossible for a tar implementation to prevent any
possible exploit. It is the duty of the user to act carefully.
For this reason, -secure-links has not made the default behavior. It cannot
work 100% correctly but it gives a wrong feeling of security.
- New error control tag SAMEFILE allows to check for links where source/destination
point to the same file.
- Added SECURITY & LSECURITY error control to the man page.
- Fixed a bug that caused star to use a wrong length for extended TAR headers.
As a result if this bug, a wrong UTF-8 -> ISO-8859 coding could occur.
- Added a test that prevents that star would remove a file if the tar archive
contains a had link to the file itself.
0 -rw-r--r-- root/root Jul 25 20:06 2003 f1
0 Hrw-r--r-- root/root Jul 25 20:06 2003 f1 link to f1
If the archive is hand crafted and the file itself has been removed,
it would have been possible to remove close to any file if the caller of star
has the needed permissions.
- Added a test that check whether source/destination are the same when copying
a instead of making a hard link. This prevents star from removing the file
by accident.
- New option -s replstr implements 'ed' like pattern substitutions
as documented for 'pax'.
- Intercative (manual) renaming now allows longer file names than 100
characters to be typed in.
- -p does no longer imply -dirmode if in create mode. If you like to
have the old behavior, call star -c -dirmode .... instead
- New option -copy
This option allows three new operation modes:
- Without other commands:
Copying files and directory trees to a target directory
in a very efficient way.
The last file type argument is a target directory.
star -copy -time -sparse -acl -p source_dir | files target_dir
- In conjunction with the -diff command:
Comparing two directory trees in a very efficient way
The last file type argument is a target directory.
star -copy -time -sparse -acl -diff -v diffopts=... source_dir | files target_dir
- In conjunction with the -t command:
A way to get a one pass tar like listing.
In this case all file type arguments are use to list.
star -copy -tv source_dir | files
- Several small bugs in the FIFO have been fixed. These bugs did only affect the
new -copy mode.
- Debugging via command line arguments (rmt debugfile) has been disabled
for security reasons.
- Allow shorthands for
star -copy -t -v -> star -ctv
star -copy -diff -> star -c -diff
- Do not try to get current working directory if star -tv -C dir
has been called
- Make FIFO code mark the sync pipe files "close on exec" so a bzip/gzip
will not inherit them and prevent the FIFO from detecting the death
of the other side
- Call on_comerr() with fifo_exit() routine so an arborted star will not
leave the FIFO process and the bzip/gzip process after it did die.
- Call new volume script with two parameters:
1 the next volume number
2 the file name of the next volume
- Better man page for -no-fsync now better describes Linux kernel implementation
design flaws (that make programs that like to grant data integrity slow),
and gives hints when using -no-fsync may speed up things on Linux on expense
of data integrity.
- Better man page for for tar archive format description
- New command 'spax' implements a POSIX.1 pax style command line interface
on top of star. The command is basically SUSv2/UNIX-98
The following extensions from SVSv3 / POSIX.1-2001 are implemented:
- The options -H and -L
- Support for the PSOX.1-2001 extended TAR header format
called 'pax'. This makes 'spax' the first pax like command that
supports this infinitely extensible and highly portable archive
format.
The following limitations currently apply:
- No support for the -l option (create hard links if possible
in copy mode).
- The privileges option -pa is ignored
- The privileges option -pm will not only not set the mtime
but also not set the atime in extract and copy mode.
- without -po SUID/SGID permission bits are not masked off.
- Pattern matching for command line type args is not yet POSIX/PAX
compliant. It should emulate the shell's way of matching file names
where '/' is not part of the pattern and each path name component
needs to match separately, but it treats a filename like an unstructured
piece of text.
Although there are several limitations, spax should be POSIX compliant enough
for the everyday work.
Rationale: spax only implements a limited set of options to grant best POSIX
compliance. Anything you can do with spax may also be done with star. To be able
to do this, a new set of options has been added to star for this reason.
- New options in star:
-pax-ls Switch to POSIX ls/pax style listing
-pax-match Allow patterns in file type arguments
-pax-c Invert matching from file type arguments
-pax-n Allow only one match per pattern
-pax-i PAX style interactive mode
-pax-p PAX style privileges option setup
-pax-L Follow all symbolic links (PAX/POSIX style)
-pax-H Follow cmd line symbolic links (PAX/POSIX style)
- list= option now also works on -copy mode
- Support for POSIX.1-1988 CPIO archive format.
Two new formats are available:
cpio The POSIX.1-1988 CPIO format
odc This is what SYSvR4 cpio -Hodc understands
Unfortunately SYSvR4 cpio is not fully POSIX.1-1988
compliant.
- Support for SYSvR4 CPIO ASC and ACRC formats
(cpio -o -c or cpio -o -Hcrc).
The checksum with -Hcrc is ignored in extract mode.
The star header format names are:
asc Equals to what SYSvR4 cpio -c creates
crc Equals to what SYSvR4 cpio -Hcrc creates
- Limited extract support for UNIX.V7 binary CPIO archives.
- Do not do sparse file checking in -diff mode if the archive
format does not support to archive sparse files.
- Fixed a typo on the property set up that caused star to print
incorrect bad checksum messages in some cases.
- First support for Linux extended file attributes.
Thanks to Andreas Grünbacher <[email protected]> for the
first implementation.
Note that this implementation is not generic enough to cover
more general Extended File Attribute implementations as found on
Solaris. The current implementation thus may go away in the future
if star implements support for Solaris Extended File Attributes.