Interesting FAQs & lessons from building ETL pipelines related to spark, dataframe, spark sql, configurations.
-
posts
-
Spark related quickies for building ETL pipelines
-
With an input address, what is your Region, Planning Area & Subzone classified by URA Singapore?
URA release a masterplan every few years and boundaries might change. With property transactions data coming in everyday, how do you identify an address is categorised under the right Planning Area & Region quickly and accurately?
-
Singaporean Household Income: Where Do You Stand in your Planning Area?
Ever wonder how much your monthly household income at your area is compared to another area? (Eg. Bedok vs Jurong) This work is inspired by a seedly article and you may find the code in my github repository.
-
Bubble Plots to Visualize Singapore EC & Condos rents figure
Embed an interactive map onto any website! This is in continuation of the previous dataset created in the previous post. Create a bubble plot map using R to visualise transactions data!
-
Exploring Singapore private property transactions with URA API Part 3
Perform data wrangling using python to create useful datasets for visualising property transactions! This is in continuation of the URA API series.
-
Exploring Singapore private property transactions with URA API Part 2
Unlocking data insights based on the rental figures of private non-landed residential properties in Singapore! This is in continuation of the URA API series
-
Exploring Singapore private property transactions with URA API Part 1
A quick tutorial on how to perform an API call using python without a separate IDE. API source can be found over here and you may find the stored in my github repository as well.