Daim duab qhia chaw thiab cov khoom txo qis ua haujlwm li cas?

Cov txheej txheem:

Daim duab qhia chaw thiab cov khoom txo qis ua haujlwm li cas?
Daim duab qhia chaw thiab cov khoom txo qis ua haujlwm li cas?

Video: Daim duab qhia chaw thiab cov khoom txo qis ua haujlwm li cas?

Video: Daim duab qhia chaw thiab cov khoom txo qis ua haujlwm li cas?
Video: Saib Mis paub Poj niam (hluas nkauj) tus yam ntxwv lub siab 2024, Hlis ntuj nqeg
Anonim

Qhov tso tawm ntawm Mapper lossis daim ntawv qhia txoj haujlwm (tus nqi tseem ceeb) yog cov lus nkag rau lub Reducer Tus reducer tau txais khub tseem ceeb-tus nqi los ntawm ntau txoj haujlwm hauv daim ntawv qhia. Tom qab ntawd, tus reducer aggregates cov ntaub ntawv nruab nrab tuples (intermediate key-value khub) rau hauv ib pawg me me ntawm tuples lossis tus nqi tseem ceeb uas yog qhov kawg tso tawm.

daim duab qhia thiab cov txo qis ua dab tsi?

Hadoop Mapper yog lub luag haujlwm lossis haujlwm uas yog siv txhawm rau ua txhua cov ntaub ntawv tawm tswv yim los ntawm cov ntaub ntawv thiab tsim cov khoom tso tawm uas ua haujlwm raws li cov tswv yim rau Reducer Nws tsim cov zis los ntawm kev rov qab tshiab key-value khub. … Tus mapper kuj tsim qee cov ntaub ntawv me me thaum ua cov ntaub ntawv tawm tswv yim raws li tus khub tseem ceeb.

Dab tsi yog qhov txawv ntawm mapper thiab reducer?

Dab tsi yog qhov txawv ntawm Mapper thiab Reducer? Mapper txoj hauj lwm yog thawj theem ntawm kev ua haujlwm uas ua txhua cov ntaub ntawv tawm tswv yim(los ntawm RecordReader) thiab tsim ib qho tseem ceeb ntawm tus nqi nruab nrab. Kev txo qis yog hu ua cais rau txhua tus yuam sij / tus nqi teev npe khub.

Koj suav pes tsawg tus mappers thiab txo qis li cas?

Nws nyob ntawm seb muaj pes tsawg cores thiab koj nco npaum li cas ntawm txhua tus qhev. Feem ntau, ib tus mapper yuav tsum tau 1 txog 1.5 cores ntawm processors Yog li yog tias koj muaj 15 cores ces ib tus tuaj yeem khiav 10 Mappers ib Node. Yog li yog tias koj muaj 100 cov ntaub ntawv nodes hauv Hadoop Cluster ces ib tus tuaj yeem khiav 1000 Mappers hauv pawg.

Mapper ua haujlwm li cas?

Mapper yog ib txoj haujlwm uas txheej txheem cov ntaub ntawv nkagTus mapper ua cov ntaub ntawv thiab tsim ntau cov ntaub ntawv me me. Cov tswv yim rau lub mapper muaj nuj nqi yog nyob rau hauv daim ntawv ntawm (qhov tseem ceeb, tus nqi) khub, txawm tias cov tswv yim rau ib tug MapReduce qhov kev pab cuam yog ib cov ntaub ntawv los yog directory (uas yog khaws cia nyob rau hauv lub HDFS).

Pom zoo: