Why does tar appear to skip file contents when output file is /dev/null? The 2019 Stack Overflow Developer Survey Results Are In Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern) 2019 Community Moderator Election ResultsWhy is my tar file bigger than its contents?mv a file to /dev/null breaks dev/nulltar: /dev/st0: Cannot write: Input/output errorMeaning of `cat /dev/null > file`Output file generated by tarWhy does my tar not work?What happens when you write to /dev/null? What’s the point?Why is /dev/null a file? Why isn't its function implemented as a simple program?Make tar from /dev/stdin fileCompare file compression methods by sending to /dev/null
Why is Captain Marvel translated as male in Portugal?
What is this lever in Argentinian toilets?
Is there a writing software that you can sort scenes like slides in PowerPoint?
Didn't get enough time to take a Coding Test - what to do now?
How is simplicity better than precision and clarity in prose?
Is there a trick to getting spices to fix to nuts?
Is it ethical to upload a automatically generated paper to a non peer-reviewed site as part of a larger research?
Did God make two great lights or did He make the great light two?
Can a 1st-level character have an ability score above 18?
Is it ok to offer lower paid work as a trial period before negotiating for a full-time job?
how can a perfect fourth interval be considered either consonant or dissonant?
Do working physicists consider Newtonian mechanics to be "falsified"?
Pandas DataFrames: Create new rows with calculations across existing rows
What LEGO pieces have "real-world" functionality?
Simulating Exploding Dice
Difference between "generating set" and free product?
What do you call a plan that's an alternative plan in case your initial plan fails?
Why can't wing-mounted spoilers be used to steepen approaches?
The following signatures were invalid: EXPKEYSIG 1397BC53640DB551
rotate text in posterbox
"... to apply for a visa" or "... and applied for a visa"?
Did the UK government pay "millions and millions of dollars" to try to snag Julian Assange?
Mortgage adviser recommends a longer term than necessary combined with overpayments
How does ice melt when immersed in water?
Why does tar appear to skip file contents when output file is /dev/null?
The 2019 Stack Overflow Developer Survey Results Are In
Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern)
2019 Community Moderator Election ResultsWhy is my tar file bigger than its contents?mv a file to /dev/null breaks dev/nulltar: /dev/st0: Cannot write: Input/output errorMeaning of `cat /dev/null > file`Output file generated by tarWhy does my tar not work?What happens when you write to /dev/null? What’s the point?Why is /dev/null a file? Why isn't its function implemented as a simple program?Make tar from /dev/stdin fileCompare file compression methods by sending to /dev/null
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
I have a directory with over 400 GiB of data in it. I wanted to check that all the files can be read without errors, so a simple way I thought of was to tar it into /dev/null. But instead I see the following behavior:
$ time tar cf /dev/null .
real 0m4.387s
user 0m3.462s
sys 0m0.185s
$ time tar cf - . > /dev/null
real 0m3.130s
user 0m3.091s
sys 0m0.035s
$ time tar cf - . | cat > /dev/null
^C
real 10m32.985s
user 0m1.942s
sys 0m33.764s
The third command above was forcibly stopped by Ctrl+C after having run for quite long already. Moreover, while the first two commands were working, activity indicator of the storage device containing . was nearly always idle. With the third command the indicator is constantly lit up, meaning extreme busyness.
So it seems that, when tar is able to find out that its output file is /dev/null, i.e. when /dev/null is directly opened to have the file handle which tar writes to, file body appears skipped. (Adding v option to tar does print all the files in the directory being tar'red.)
So I wonder, why is this so? Is it some kind of optimization? If yes, then why would tar even want to do such a dubious optimization for such a special case?
I'm using GNU tar 1.26 with glibc 2.27 on Linux 4.14.105 amd64.
tar null
asked 28 mins ago
RuslanRuslan
1,3891427
add a comment |
I have a directory with over 400 GiB of data in it. I wanted to check that all the files can be read without errors, so a simple way I thought of was to tar it into /dev/null. But instead I see the following behavior:
$ time tar cf /dev/null .
real 0m4.387s
user 0m3.462s
sys 0m0.185s
$ time tar cf - . > /dev/null
real 0m3.130s
user 0m3.091s
sys 0m0.035s
$ time tar cf - . | cat > /dev/null
^C
real 10m32.985s
user 0m1.942s
sys 0m33.764s
The third command above was forcibly stopped by Ctrl+C after having run for quite long already. Moreover, while the first two commands were working, activity indicator of the storage device containing . was nearly always idle. With the third command the indicator is constantly lit up, meaning extreme busyness.
So it seems that, when tar is able to find out that its output file is /dev/null, i.e. when /dev/null is directly opened to have the file handle which tar writes to, file body appears skipped. (Adding v option to tar does print all the files in the directory being tar'red.)
So I wonder, why is this so? Is it some kind of optimization? If yes, then why would tar even want to do such a dubious optimization for such a special case?
I'm using GNU tar 1.26 with glibc 2.27 on Linux 4.14.105 amd64.
tar null
asked 28 mins ago
RuslanRuslan
1,3891427
add a comment |
I have a directory with over 400 GiB of data in it. I wanted to check that all the files can be read without errors, so a simple way I thought of was to tar it into /dev/null. But instead I see the following behavior:
$ time tar cf /dev/null .
real 0m4.387s
user 0m3.462s
sys 0m0.185s
$ time tar cf - . > /dev/null
real 0m3.130s
user 0m3.091s
sys 0m0.035s
$ time tar cf - . | cat > /dev/null
^C
real 10m32.985s
user 0m1.942s
sys 0m33.764s
The third command above was forcibly stopped by Ctrl+C after having run for quite long already. Moreover, while the first two commands were working, activity indicator of the storage device containing . was nearly always idle. With the third command the indicator is constantly lit up, meaning extreme busyness.
So it seems that, when tar is able to find out that its output file is /dev/null, i.e. when /dev/null is directly opened to have the file handle which tar writes to, file body appears skipped. (Adding v option to tar does print all the files in the directory being tar'red.)
So I wonder, why is this so? Is it some kind of optimization? If yes, then why would tar even want to do such a dubious optimization for such a special case?
I'm using GNU tar 1.26 with glibc 2.27 on Linux 4.14.105 amd64.
tar null
asked 28 mins ago
RuslanRuslan
1,3891427
I have a directory with over 400 GiB of data in it. I wanted to check that all the files can be read without errors, so a simple way I thought of was to tar it into /dev/null. But instead I see the following behavior:
$ time tar cf /dev/null .
real 0m4.387s
user 0m3.462s
sys 0m0.185s
$ time tar cf - . > /dev/null
real 0m3.130s
user 0m3.091s
sys 0m0.035s
$ time tar cf - . | cat > /dev/null
^C
real 10m32.985s
user 0m1.942s
sys 0m33.764s
The third command above was forcibly stopped by Ctrl+C after having run for quite long already. Moreover, while the first two commands were working, activity indicator of the storage device containing . was nearly always idle. With the third command the indicator is constantly lit up, meaning extreme busyness.
So it seems that, when tar is able to find out that its output file is /dev/null, i.e. when /dev/null is directly opened to have the file handle which tar writes to, file body appears skipped. (Adding v option to tar does print all the files in the directory being tar'red.)
So I wonder, why is this so? Is it some kind of optimization? If yes, then why would tar even want to do such a dubious optimization for such a special case?
I'm using GNU tar 1.26 with glibc 2.27 on Linux 4.14.105 amd64.
tar null
tar null
asked 28 mins ago
RuslanRuslan
1,3891427
asked 28 mins ago
RuslanRuslan
1,3891427
asked 28 mins ago
RuslanRuslan
1,3891427
asked 28 mins ago
RuslanRuslan
1,3891427
asked 28 mins ago
RuslanRuslan
1,3891427
1,3891427
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
It is a documented optimization:
When the archive is being created to
/dev/null, GNU tar tries to
minimize input and output operations. The Amanda backup system, when
used with GNU tar, has an initial sizing pass which uses this feature.
answered 10 mins ago
murumuru
37.6k589165
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f512362%2fwhy-does-tar-appear-to-skip-file-contents-when-output-file-is-dev-null%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
It is a documented optimization:
When the archive is being created to
/dev/null, GNU tar tries to
minimize input and output operations. The Amanda backup system, when
used with GNU tar, has an initial sizing pass which uses this feature.
answered 10 mins ago
murumuru
37.6k589165
add a comment |
It is a documented optimization:
When the archive is being created to
/dev/null, GNU tar tries to
minimize input and output operations. The Amanda backup system, when
used with GNU tar, has an initial sizing pass which uses this feature.
answered 10 mins ago
murumuru
37.6k589165
add a comment |
It is a documented optimization:
When the archive is being created to
/dev/null, GNU tar tries to
minimize input and output operations. The Amanda backup system, when
used with GNU tar, has an initial sizing pass which uses this feature.
answered 10 mins ago
murumuru
37.6k589165
It is a documented optimization:
When the archive is being created to
/dev/null, GNU tar tries to
minimize input and output operations. The Amanda backup system, when
used with GNU tar, has an initial sizing pass which uses this feature.
answered 10 mins ago
murumuru
37.6k589165
answered 10 mins ago
murumuru
37.6k589165
answered 10 mins ago
murumuru
37.6k589165
answered 10 mins ago
murumuru
37.6k589165
37.6k589165
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f512362%2fwhy-does-tar-appear-to-skip-file-contents-when-output-file-is-dev-null%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
