Great video, thank you. DeltaJSON is really useful for devops working with JSON in AWS, it does compare, merge and graft via API service.
@DataEngUncomplicated
2 жыл бұрын
Cool I'll check it out, thanks mark!
@EagleHunt
2 жыл бұрын
Hi, I am trying to follow along but at 3:18 I only have the CloudWatch Logs option. I cannot select s3, what should I do? Any help would be greatly appreciated!
@DataEngUncomplicated
2 жыл бұрын
Hi, it sounds like you are missing the required iam permissions for your lambda role. Make sure to update it for the permissions you need to s3
@georgelza
Жыл бұрын
Personal request, do you have an example where the JSON is published onto a Kafka topic ?
@DataEngUncomplicated
Жыл бұрын
Hi George, no unfortunately I haven't worked with Kafka
@georgelza
Жыл бұрын
@@DataEngUncomplicated bummer... it's such an important building block these days in data streaming, and Lambda is such a nice "publisher" from the AWS ecosystem, thanks. back to Google searching.
@georgelza
Жыл бұрын
@@DataEngUncomplicated sorry, another question, I got a potential inbound stream, .json.gz that i first need to uncompress. , the resulting .json file could then have up to 1million records.... comment from viewpoint of awswrangler handling this and Lambda handling this ? At any time i might have ,multiple of these files being dropped into the S3 bucket.
@DataEngUncomplicated
Жыл бұрын
I have handled more than 1 million records using aws wrangler. You will need to increase the memory from the default to handle it though. Keep in mind, if your lambda functions is doing complex transformations and creating many data frames in memory, it will eat up more memory and you could run out of memory.
@georgelza
Жыл бұрын
@@DataEngUncomplicated all that i need to do is loop through the doc and post them as individual messages onto a kafka topic, so little processing.
@tekkemganesh9424
Жыл бұрын
Can we read image from s3 bucket and integrate it with Amazon eventbridge and send it to the lambda function to trigger can any one helpme??
@DataEngUncomplicated
Жыл бұрын
Yes, when you set up Eventbridge, the key and bucket should be in the metadata, you can write a lambda function which can then read that image into your lambda function
Пікірлер: 15