Excellent! Critically, it was more informative than the AWS docs ;)
@FRUXT
3 ай бұрын
buy a micro please...
@udaynayak4788
Жыл бұрын
it was really helpful, thank you for sharing.
@ajitkalawade9880
9 ай бұрын
How we can update stack when we modify our glue jobs how it will reflect automatically in this spark history logs..please help @public cloud
@subhasisdalal2913
Жыл бұрын
Hello, Thanks for posting the video. I am trying to do the same in my project. However, I have a question at 7:28 of the video. How do we know in S3 (Spark-ui-logs), which file is for which job run. Is there any mapping of the log to the glue job. Please help.
@Publicloud
Жыл бұрын
From the Glue Driver Logs (under monitoring > job run) we can see the Spark UI Log file names; typically we may not need to identify the spark ui log file names for the job run as once we access the Spark Web UI, the job run and associated log details will get mapped automatically and available visually for analysis.
@sairamnagarajan8289
Жыл бұрын
Thanks for your effort. One quick question. Why the first glue job run log was not loaded in spark History server UI.?
@AG-im3dz
Жыл бұрын
Because the server was created AFTER the first job was ran
@abhishekdubey-p9n
Жыл бұрын
An error occurred while calling o227.pyWriteDynamicFrame. Unable to parse file
@Publicloud
Жыл бұрын
Please check the source /target file properties in Glue, something around format/delimiter may not be set correctly.
@abhishekdubey-p9n
Жыл бұрын
@@Publicloud the error is coming because i want to convert multiple CSV files to parquet from the same folder if i use single file i can achieve the same pls help me out ...
Пікірлер: 11