hadoop - Flume: Not able to fix sink output file size -


i trying fix sink output file size. i.e trying 128 mb each output file. tried several mechanism ( rollinterval,rollcount,rollsize) did not desired output. not getting consistently 128 mb files. getting few 128 mb files later on files generated different sizes 30,40 45 mb etc. , lot of newly created files opens , remains @ .tmp state. idea?

i don't think possible create 128mb size file.if flume aggregate data of random size (i mean not constant size) or data of constant size not multiple of requested size, create files of lower size 128.

i guess need have constant flow of small data , have tmp file unless 1 filled (is 128mb large). if monitoring directories files have multiples of 128 instead of have part file of lower size.

hope correctly understood problem.


Comments