Mageswaran D – Medium

Mageswaran D

Mageswaran D

How to keep Cloud cost under control?

Collect the hardware metrics (CPU/memory utilization, disk I/O)

Apr 16, 2022

How to keep Cloud cost under control?

Apr 16, 2022

Mageswaran D

2022 : No FileSystem for scheme: s3?

How many times did you face this error? S3 or S3a schema not found error while working with AWS S3 + Spark outside the EMR?

Mar 24, 2022

2022 : No FileSystem for scheme: s3?

Mar 24, 2022

Mageswaran D

PySpark Practice Problems

How to transform array of arrays into columns?

Jan 2, 2022

PySpark Practice Problems

Jan 2, 2022

Mageswaran D

Dynamic Programming Patterns to Ace Interviews

Following materials are taken from freecodeCamp.org Youtube video course https://www.youtube.com/watch?v=oBt53YbR9Kk strongly recommend to…

Dec 15, 2021

Dynamic Programming Patterns to Ace Interviews

Dec 15, 2021

Mageswaran D

What if I wanted to submit remote PySpark jobs to AWS EMR without worrying about library dependen…

Typically Spark cluster is used to run ETL jobs or some streaming jobs, where everything is managed by developers for developers, where…

Dec 8, 2021

What if I wanted to submit remote PySpark jobs to AWS EMR without worrying about library dependen…

Dec 8, 2021

Mageswaran D

NLP: What it takes to design a full stack DeepLearning based Receipts form filling system using NER?

Online Colab Notebook for model training.

Nov 29, 2021

NLP: What it takes to design a full stack DeepLearning based Receipts form filling system using NER?

Nov 29, 2021

Mageswaran D

NLP: What it takes to model Google search suggestions or auto complete?

Dataset : google_wellformed_query 25K examples

Nov 10, 2021

NLP: What it takes to model Google search suggestions or auto complete?

Nov 10, 2021

Mageswaran D

How to build custom NER HuggingFace dataset for receipts and train with HuggingFace Transformers…

Disclaimer: It is assumed that you have some working knowledge in Hugging face library and datasets library, to begin with.

Oct 28, 2021

How to build custom NER HuggingFace dataset for receipts and train with HuggingFace Transformers…

Oct 28, 2021

Mageswaran D

A dive into Apache Spark Parquet Reader for small size files

While working part of my current project, I was asked how Spark reads Parquet files and how does it achieves the parallelisation. Very…

Oct 26, 2021

A dive into Apache Spark Parquet Reader for small size files

Oct 26, 2021

Mageswaran D

Some interesting interview Q&A around “randomness”

1 How to sample a random Element from an infinite stream?

May 24, 2021

May 24, 2021

Mageswaran D

Mageswaran D

A simple guy in pursuit of of AI and Deep Learning with Big Data tools :) @ https://www.linkedin.com/in/mageswaran1989/

Following

Help
Status
About
Careers
Press
Blog
Privacy
Terms
Text to speech
Teams