Can I split compressed files?

I am using bzip to compress large files. My custom InputFormat returns true for isSplitable but I get failures with large input files.

BZip2 splitting support was added to Apache Hadoop in the 0.21.0 release but I am running 0.20. Is that my problem?

bzip compression split

asked 22 Jun '11, 08:57

FAQ ♦
147●36●37●39
accept rate: 0%

One Answer:

1	WIth MapR, compression is built in and is entirely transparent. This means that your compressed inputs are split just like any other file. No action is required on your part. link answered 30 Jun '11, 18:51 TedDunning ♦♦ 2.4k●3●15 accept rate: 28%

Your answer

toggle preview

community wiki

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

*italic* or __italic__
**bold** or __bold__
link:[text](http://url.com/ "title")
image?![alt text](/path/img.jpg "title")
numbered list: 1. Foo 2. Bar
to add a line break simply add two spaces to where you would like the new line to be.
basic HTML tags are also supported

Tags:

Asked: 22 Jun '11, 08:57

Seen: 713 times

Last updated: 30 Jun '11, 18:51