About midway through the 2nd week of an 8 week project. I'm working for a large insurance company located in Downtown Boston. What technologies am I working on for this project? I work on Operational Reports for the Actuarial department. They have a source database, a team that gets the data into AWS Data Lake, Hadoop Hive tables. We connect using an IDE called DBVisualizer and write custom SQL statements. Also some Power BI and Tableau development.
I spent some time researching Hive optimization techniques. They have partitioning, bucketing, indexing, writing better SQL code, but they also have other options. They recommend using Sort By rather than Order by, specify the order of your Group By fields, avoid nested Sub-Queries, use Between rather than <= and >=.
Found a few good links I read:
Basically its full life cycle report development. Gather specs, map the fields, write the queries, validate the data with the Business, deploy to production, document, maintain and enhance. I've worked for an Insurance company before, so I understand the basic concepts such as Inforce, Written Premium, Earned Premium, Claim Payments, etc.
I do enjoy working in different regions with different clients, people, projects, challenges, scenery and weather. I guess that's one good thing about consulting, never the same day twice.
And there you have it~!
I signed up for the Hortonworks Certified Associate exam last Thursday. Figured if I sign up, I'd have to take the test. And if I tak...
Data becomes information. Information adds value if used properly to align business practices, streamline processes with net result of incr...
Data is the new oil. Sort of a good analogy. Except new oil is constantly required. And there is only so many oil wells on the planet. A...
What do you want to do when you grow up. For some of us, we still haven't decided. After close to 50 years. Chances are, if you chos...