Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Ask Slashdot: Changing Career From OLTP To OLAP Dev

samzenpus posted more than 2 years ago | from the switching-teams dept.

Programming 129

First time accepted submitter xby2_arch writes "After spending over 12 years writing OLTP applications (Java EE/JDBC/ORMs), I decided to dabble in the OLAP world. I had decent DB skills, considering most of my previous projects had involved data modeling and coding using Stored Procs, etc. Yet I hadn't designed or implemented any dimensional databases. Luckily for me, I had enough relevant domain knowledge to land a developer job in a data warehousing project. The work was enjoyable enough that it motivated me to spend that extra time and effort I needed to cope with the different dynamics of coding in the OLAP realm. In my past life, data volumes weren't the primary concern (instead, transaction volumes were), here, everything was about data. ETL/Integrations present another set of problems you generally skirt in a typical web/app-tier developer role. All in all, it turned out to be a non-trivial, yet worthwhile transition. I am certain that there are plenty of seasoned developers out there who plan to make a similar move (or have made already), who see data as the next chapter in their careers evolving toward becoming Enterprise Architects. I want to hear what's holding them back, or what helped them move forward. What should be considered a prerequisite to make this switch, and what are the risks, etc.?"

cancel ×

129 comments

Sorry! There are no comments related to the filter you selected.

Welcome to Slashdot (5, Funny)

Anonymous Coward | more than 2 years ago | (#38707456)

Please stand by while the trolls try to figure out how to respond to your question.

In the meantime, you should expect about 50 posts by PHP developers asking what a stored proc is, and suggesting you move to RoR if you want an ORM.

Re:Welcome to Slashdot (4, Funny)

hedwards (940851) | more than 2 years ago | (#38707476)

The OP should quit his job.

Re:Welcome to Slashdot (1)

defcon-11 (2181232) | more than 2 years ago | (#38708624)

Anyone who works at a business that hires employees with the title 'Enterprise Architect' should probably quit their job.

Re:Welcome to Slashdot (1)

cayenne8 (626475) | more than 2 years ago | (#38710132)

Anyone who works at a business that hires employees with the title 'Enterprise Architect' should probably quit their job.

Why?

Those positions usually pay quite well...

Re:Welcome to Slashdot (1)

alphatel (1450715) | more than 2 years ago | (#38707480)

On behalf of the trolls I would like to ask, why are you anonymous?

Re:Welcome to Slashdot (1)

Anonymous Coward | more than 2 years ago | (#38709368)

So we can post stupid answers and not lose karma...duh.

Re:Welcome to Slashdot (-1)

Anonymous Coward | more than 2 years ago | (#38707742)

I'm a coder (or what they would have called a "hacker" in ye olden days), and what is a OLTP, OLAP, RoR or ORM?

I'm assuming it's some enterprisey consultant [thedailywtf.com] shit.

Re:Welcome to Slashdot (0)

Anonymous Coward | more than 2 years ago | (#38707792)

I think they're acronyms that you use to blind the juniors into believing that you're working hard and know what you're talking about, while you spend six months reading Slashdot and trying to get management to pay for your next jaunt to LISA.

ooooh (1)

unity100 (970058) | more than 2 years ago | (#38707890)

50 posts by PHP developers asking what a stored proc is

we feel offended. as if a thousand developers cried in agony because someone who is in a different field thinks that his field is more elite than ours. oh the humanity !

hear ye ! hear ye ! person in random field thinks those in another field are less important and knowledgeable - and their work too.

Re:ooooh (1)

Anonymous Coward | more than 2 years ago | (#38708030)

The irony is that in 15 years, if you keep learning and advancing in the art of software development, you will be making the exact same post as the GP.

nope. (3, Insightful)

unity100 (970058) | more than 2 years ago | (#38708342)

Actually i matured 5 years or so ago, after spending some 10 years in the delusion of thinking that the fields i was in were more 'elite' than the others. they were different fields than programming. but, the core concept of 'our work is tougher and more elite - other people are not as elite as us' nonsense was the same.

i grew over it.

Re:nope. (0)

Anonymous Coward | more than 2 years ago | (#38708374)

Wish HR morons and head hunters would too.

Re:nope. (1)

unity100 (970058) | more than 2 years ago | (#38709164)

human resources ......... they constitute a total fantasy land in themselves.

Re:ooooh (1)

Surt (22457) | more than 2 years ago | (#38708280)

And yet, PHPs raison d'etre is that real frameworks were too hard to figure out.
And ever after, as people struggled to deploy real applications on it, it has evolved towards being a real platform. Now that they're half-way there, it's now half as complex to learn as a real framework. Quelle surprise.

Re:ooooh (1)

unity100 (970058) | more than 2 years ago | (#38708578)

And yet, PHPs raison d'etre is that real frameworks were too hard to figure out

wrong. the main reason php came into being was the need for a framework that was fast to develop on, and non proprietary. asp was there otherwise. cgi was also there, but it was not flexible.

'people struggled to deploy real applications on it' ? you probably are joking. or you havent at all delved into what php has become. flickr, facebook, yahoo ... if you are not counting what runs on these sites as 'real applications', then you are probably referring to non web related or niche applications that do other things than serving data out to users on the internet. quelle surprise ! - there are endless numbers of php libraries that do very queer things if you compile php with too.

but who am i talking to. you are just going to keep yourself in your narrow world where 'real applications' reside.

Define, please? (0)

danaris (525051) | more than 2 years ago | (#38707466)

So, um, what the crap are OLTP and OLAP?

(And yes, this marks me as hopelessly undereducated, and obviously a fool who doesn't know anything about Real Programming. So sue me. Just tell us what they are, too, please?)

Dan Aris

Re:Define, please? (4, Informative)

betterunixthanunix (980855) | more than 2 years ago | (#38707512)

You know, we can always check Wikipedia for definitions:

https://en.wikipedia.org/wiki/OLTP [wikipedia.org]
https://en.wikipedia.org/wiki/OLAP [wikipedia.org]

It might have been nice if the editors had included these links in the summary, but it is not as though you do not know how to use Wikipedia.

Re:Define, please? (3, Insightful)

FoolishOwl (1698506) | more than 2 years ago | (#38707814)

While it's a fair point that Wikipedia is an obvious starting point, I would have to say that the article on OLAP seems to suggest that OLTP is a synonym for OLAP, and the article on OLAP is short but dense. I'm left with an impression that there's a distinction being made between two approaches for constructing end-user interfaces to databases, but what the distinction between OLTP and OLAP is still unclear to me, and I don't have any idea why the distinction would be so significant as to constitute a career transition.

Re:Define, please? (5, Informative)

Anonymous Coward | more than 2 years ago | (#38708300)

If it's on the main page the target audience is pretty general, so you really shouldn't have to check Wikipedia. You should ALWAYS tell people what acronyms mean before using them exclusively. And if you have time enough to be an arrogant *&$# while posting links to Wikipedia you could also take the time to spell out what the acronyms mean. At the very least give the long form, e.g. Online Transaction Processing (OLTP) [wikipedia.org] , Online Analytical Processing (OLAP) [wikipedia.org] . See it's not that fucking hard.

And since I am taking the time to rant (and because any technical article in Wikipedia gets hijacked by propellor heads who like to inject as much as of their industry specific double speak so that they sound important and a layman can't get at least an understandable overview in the summary):

OLTP - Online Transaction Processing. An application/database designed for larger, equal, or nearly equal volumes of database inserts, updates, and deletes, as there are reads. The database is generally more (and often highly) normalized, meaning that there is less data duplication across tables and/or within tables.
OLAP - Online Analytical Processing. Essentially a data warehouse designed for larger volumes of reads than there will be inserts, updates, or deletes (often relatively very, very few deletes). Less normalized meaning that there may be duplicated data across and/or within tables in order to increase query speed at the cost of possible consistency issues due to the data duplication.

Re:Define, please? (4, Informative)

kurthr (30155) | more than 2 years ago | (#38708406)

OLTP - Online Transaction Processing. An application/database designed for larger, equal, or nearly equal volumes of database inserts, updates, and deletes, as there are reads. The database is generally more (and often highly) normalized, meaning that there is less data duplication across tables and/or within tables.
OLAP - Online Analytical Processing. Essentially a data warehouse designed for larger volumes of reads than there will be inserts, updates, or deletes (often relatively very, very few deletes). Less normalized meaning that there may be duplicated data across and/or within tables in order to increase query speed at the cost of possible consistency issues due to the data duplication.

Mark this informative comment the hell up!
And smash my Karma if you want to, but the (four letter) acronyms really didn't help the article much.

Re:Define, please? (0)

Anonymous Coward | more than 2 years ago | (#38709950)

OTOH, if you don't know what the acronyms mean, you're not exactly going to have anything to say about the question.

Other than to troll people for not spelling out the thing you're ignorant of.

Re:Define, please? (1)

SplashMyBandit (1543257) | more than 2 years ago | (#38708606)

Sometimes though when people abbreviate you can get more than one abbreviation that fits the context. This is worse the more you know, as there are more ways you can see an abbreviation could possibly fit. The solution, of course, is for the original poster (OP) to define the abbreviation the first time they use it. That's a good rule of thumb for technical writing, and just as good on Slashdot too :)

Re:Define, please? (1)

Anonymous Coward | more than 2 years ago | (#38707686)

OLTP is designing a database schema that is normalized for more robust and maintainable data storage. OLAP is denormalized to make it easier to produce pretty charts that managment can understand without having to think too hard about their select statements.

Re:Define, please? (0)

Anonymous Coward | more than 2 years ago | (#38708202)

OLTP is designing a database schema that is normalized for more robust and maintainable data storage. OLAP is denormalized to make it easier to produce pretty charts that managment can understand without having to think too hard about their select statements.

I misread "denormalized" as "demoralized" at first, and interestingly enough, the message is still the same.

Re:Define, please? (5, Informative)

Koen Lefever (2543028) | more than 2 years ago | (#38707694)

OLTP: Online Transaction Processing: you buy a ticket, you want it immediatly. The seller types it in the computer and prints your ticket, a database checked if there were free seats and immediatly reserves one. "Immediatly" is the important word, the customer is not waiting. OLAP: Online Analytical Processing: How many seats from the US to EMEA have been sold by that kind of sellers with such produyt code. Managements wants the results by the end of the month, it is OK if the query runs a couple of hours/days. Many real life systems contain two databases, one tuned for speed (containing only the current tickets) and one for reference later (containing all tickets sold in the past n months/years). The difference between them is database tuning and SQL tuning. All the rest (such as path to architect: yeah, the more different systems you know, the more chance you will have to design new ones) is hype.

Re:Define, please? (3, Informative)

UnknowingFool (672806) | more than 2 years ago | (#38707820)

The difference between them is database tuning and SQL tuning. All the rest (such as path to architect: yeah, the more different systems you know, the more chance you will have to design new ones) is hype.

That's like saying web coding is the same as application coding and the main difference is just languages used. There is a huge difference in the goals and methodology of OLTP and OLAP. DB and SQL tuning is a small part of the larger scheme of things. Take for instance your example of tickets; you can get an OLTP to show how many seats from US to EMEA by seller. It may not be very easy especially as you add conditions (by quarter, by day of week of travelled, by day of week sold, etc). In OLAP these questions become important and you have to design your DB structure to accommodate them and other queries. It also matters how this information is reported by tool as not all tools can take advantage of DB features. After all that then DB and SQL tuning comes into play.

Re:Define, please? (1)

emj (15659) | more than 2 years ago | (#38708066)

I think that was that he ment by tuning. Just the small matter of programming you know..

Re:Define, please? (1)

UnknowingFool (672806) | more than 2 years ago | (#38708584)

Tuning is optimization. Tuning is not design. In the example he gave, the OLTP database table may have SellerId, TravelDate, PurchaseDate, and CustomerId. The OLAP for his single report would have SellerId, MondayPurchases, TuesdayPurchases, etc. And that's if the OLAP architect decided to make a report into a single table. The OLAP architect may decide that the structure must accommodate many more queries. The OLAP architect may determine that a relational table isn't good enough and cubes should be used. The all queries must be written in MDX and not SQL. There is a much larger disparity in OLTP and OLAP than most people think.

Re:Define, please? (2)

secretsquirel (805445) | more than 2 years ago | (#38708798)

cubes is just a type of relational design (or at least it should be)

Re:Define, please? (1)

UnknowingFool (672806) | more than 2 years ago | (#38709668)

An OLAP cube is not a relational DB. Some cubes are built on Relational DBs like Essbase running under Oracle but OLAP Cube Servers [wikipedia.org] are often separate purchases.

Re:Define, please? (1)

Koen Lefever (2543028) | more than 2 years ago | (#38710810)

I agree with most of your points. Of course there is more to it. I apologise for not including the programmers, table designers, network folks, etc... among the non-hype, my bad. On the mixed installations I've been working on, the systems to pump data from OLTP to OLAP were for example most intricate.

The cubes etc... were done by middleware or client software, underneath it everything was relational. The tables were remodelled versions of the production systems: big item table, huge item_detail table, a bunch of reference tables with product codes, customer adresses and hour tables. The ways in which the tables were physically implemented, regarding partitions etc..., was different between OLTP & OLAP, but using the same kind of RDBMS (Oracle, Sybase, PostgreSQL, etc...).

The OLAP queries were not written by programmers. They were generated by management and analyst, using drag & drop in the client software. Manual SQL tuning of those generated queries easily results in an increase of execution speed by factor ten, from 50 hours to well below 5 hours.

I was being provocative in my formulation because the OP was in my opinion flowing a bit too much with the hype. The point I tried to make concerning the original "Ask /. question" was: switching between OLTP & OLAP is just learning some new tools and techniques, which is always a good thing - just don't listen too much to marketing when deciding what your carreer will be (unless if you want to have marketing carreer, then by all means go ahead). There is no "carreer change" from OLTP to OLAP, it is just more of the same: data in, data stored, data retreived, data out, data processed, old data purged or perhaps archived, backups taken.

Re:Define, please? (1)

Koen Lefever (2543028) | more than 2 years ago | (#38710874)

I think that was that he ment by tuning. Just the small matter of programming you know..

Absolutely.

Re:Define, please? (0)

Anonymous Coward | more than 2 years ago | (#38708592)

That's like saying web coding is the same as application coding and the main difference is just languages used.

But it is. Anyone who says otherwise is just a crap developer.

Re:Define, please? (0)

Anonymous Coward | more than 2 years ago | (#38710998)

Although you could develop your quoting method a bit. ;)

This is exactly what is wrong with the world. (0)

Anonymous Coward | more than 2 years ago | (#38707474)

FUCK. I can't tell you how many times I've tried to use software written by some ignorant fuck who thought that his knowledge of programming languages translated into knowledge of the industry he was in. Learn a language and and industry... and stick to it.

Re:This is exactly what is wrong with the world. (1)

physburn (1095481) | more than 2 years ago | (#38707840)

Industries change in size and number of staff they need. Having only one language and one industry, couldn't garantee a career. Besides its boring not learn.

As a DBA myself... (5, Funny)

Ethanol-fueled (1125189) | more than 2 years ago | (#38707494)

The first thing we did to strategize mission-critical web-readiness to expedite wireless users was leverage our dot-com ROI and aggregate robust e-markets. Being able to synthesize cutting-edge channels enabled us to unleash extensible users, which in-turn enabled us to orchestrate turn-key mindshare.

Utilizing our synergistic functionalities, we were then focused towards mesh visionary markets and envisioneering collaborative initiatives with our partners. The net result was the incubation of plug-and-play experiences to transform vertical vortals and utilize cutting-edge deliverables.

Plus, I have a large cock. That helped a lot.

Re:As a DBA myself... (3, Funny)

Hognoxious (631665) | more than 2 years ago | (#38707654)

That was so awesome I'm going to have a sex change just so I can marry you.

Re:As a DBA myself... (0, Offtopic)

Anonymous Coward | more than 2 years ago | (#38707740)

Sorry buddy. I have crushes on Snowgirl [slashdot.org] and Samantha Wright. [slashdot.org]

  Their big....brains get my meat-missle throbbing and ready to launch. I would eat out their belly-button lint and clean out their toejam with my teeth.

-- Ethanol-fueled

Re:As a DBA myself... (0)

Anonymous Coward | more than 2 years ago | (#38707704)

Yeah, but you didn't have a solution. What's the point if you don't have a solution?

Re:As a DBA myself... (2)

mike260 (224212) | more than 2 years ago | (#38707960)

The first step to a solution is always a robust solutioneering methodology.

Re:As a DBA myself... (1)

unity100 (970058) | more than 2 years ago | (#38707900)

perfect corporatespeak.

Re:As a DBA myself... (4, Funny)

todrules (882424) | more than 2 years ago | (#38708050)

-1. You forgot the cloud.

Re:As a DBA myself... (1)

Anonymous Coward | more than 2 years ago | (#38709826)

You don't need DBAs for the cloud!! You just fire up a couple more instances and blammo!

Re:As a DBA myself... (0)

Anonymous Coward | more than 2 years ago | (#38708132)

What I love about Slashdot is how you guys can masturbate to the same exact jokes over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and over and it's true, they're exactly as funny as they were the first time.

Re:As a DBA myself... (0)

Anonymous Coward | more than 2 years ago | (#38708904)

I bet you've got an ITIL or PRINCE2 certificate, don't you?

Re:As a DBA myself... (1)

jones_supa (887896) | more than 2 years ago | (#38711042)

And just to clarify, you get a PRINCE2 certificate when you complete Prince of Persia 2: The Shadow and the Flame. It's handed over by the Jordan Mechner Foundation.

Re:As a DBA myself... (1)

Mr.No (752782) | more than 2 years ago | (#38710914)

The first thing we did to strategize mission-critical web-readiness to expedite wireless users ....

Superb.

Different worlds (0)

Anonymous Coward | more than 2 years ago | (#38707496)

Man, I am glad I work in the world of startups where we just Get Shit Done.

12 years doing "OLTP"??? Just fucking shoot me.

Re:Different worlds (2)

Capt. Skinny (969540) | more than 2 years ago | (#38707914)

Meanwhile, those of us in established businesses Do Shit Right The First Time because our work is expected to hold up for years. "We Get Shit Done" is a euphemism for "We don't bother plan or learn best practices, we'll just kludge something together and fix it when it breaks."

Re:Different worlds (1)

bjourne (1034822) | more than 2 years ago | (#38709226)

And "Do Shit Right The First Time" is an euphism for "I have no fucking clue what I'm doing which is why this web gui is taking me two months to complete." :)

Re:Different worlds (0)

Anonymous Coward | more than 2 years ago | (#38709798)

The best is the enemy of the good. Some of us are good enough to do it fast and do it right. We don't fuck around posting to hip fora very much, but we're out there.

My systems handle millions of dollars a month in the electrical utility business and they do it well. I wrote version 1.0 in six weeks from a blank text editor and an empty base directory, including the framework everything was built on. Some initial decisions could have been better, but 2.0 followed six months later and we had our first application customer a couple of months after that. I was the sole author, and I ran the system for the first year. The big utility companies' systems, full of consultants and "best practices" were a constant pain in the ass to other market participants--I spent more time writing defensive code against the stupid shit that came out of their systems than doing anything else.

The project before that required an oscilloscope for debugging.

Re:Different worlds (1)

SplashMyBandit (1543257) | more than 2 years ago | (#38708626)

lol. The 12 years were probably getting *one* project out ... just kidding.

Don't post here... (0)

Anonymous Coward | more than 2 years ago | (#38707530)

..post to a board that expands uncommon acronyms.
You know, somewhere professional.

Eh? (5, Interesting)

PCM2 (4486) | more than 2 years ago | (#38707538)

So... you're saying you've already made the switch from OLTP to OLAP and you'd like to take this opportunity to gloat about it, but you'd still like to hear from other developers what they think the prerequisites are for making such a move and what has held them back from doing all the cool stuff you're doing? Or am I missing the question?

Re:Eh? (0)

Anonymous Coward | more than 2 years ago | (#38707596)

Maybe OP is making / has made the switch but would like info about whether he's making the switch as well as he should be doing, according to people who are better at OP's new job (and possibly OP's previous job) or who have more formal training in OP's new field than OP does. For example, maybe OP would like to hear from a professor, or someone who's really, really old.

Re:Eh? (4, Insightful)

vlm (69642) | more than 2 years ago | (#38707724)

My guess is its a subtle attempt at a philosophical statement that OLTP and OLAP are such wildly different problem domains that it may as well be like trying to horizontally jump from being a fluffer to being a farmer, despite them starting with the same letter and occasionally being confused, at least on /.

Personally I disagree with that viewpoint. You're still in the realm of high performance computing, blah blah blah. They have different enough needs that maybe they need to be separate groups underneath separate supervisors, or at a big enough firm, maybe separate depts, etc, but I'm not seeing a huge fundamental difference. Consider yourself lucky to be at a firm that separates those roles at all. In the last decimal place the problem solving techniques are going to be different, but I'm sure 99.9999% of it is the same, which is probably how a OLTP guy got a OLAP job without the HR filter of "must have 9245 years experience with this specific version of software" filter which theoretically caused this whole discussion. I'm sure a guy who can figure out OLTP can probably figure out OLAP rather quickly and vice versa.

Admittedly I've spent probably a hundred times more career effort on OLAP tasks than OLTP tasks, but very few firms have the luxury of full time personnel for those jobs, so I've spent most of my time doing other stuff.

If anything, I'd say take some more risks. You fail at a OLTP task and customers and sales and marketing scream and data might be lost or corrupted permanently. You fail at a OLAP task and, eh, the TPS reports are released a little later today, we'll survive... Admittedly OLAP stuff is usually a lot more complicated that OLTP, which means you're used to poor / no documentation, maybe you should be worried about doing more documentation of OLAP stuff.

Maybe another way to describe it, is fail once at OLTP and you get fired, fail repeatedly badly enough at OLAP and the company goes out of business before they figure out to fire you?

Re:Eh? (1)

guruevi (827432) | more than 2 years ago | (#38710754)

Basically, the way I most easily understand it:

OLTP provides you most of the business end of an application (it gives a structured interface to the data to read/write/modify) while OLAP provides you with the information end of an application (it gives an overview or what some may call a "dashboard" to the data which usually has limited (usually no) options for the data to be modified) which could or could not be in a feedback loop with each other.

So OLTP gives you a database for your support staff and a database for your sales people and a database to HR and they use these databases daily and modify them.

OLAP then collects the information in these databases and analyzes it (usually overnight) and (sometimes) stuffs it back into the OLTP databases (such as these types of customer have been annoyed recently, which geographical areas are trending in sales, how a new release or price point affects the market etc etc.) or generates reports to the managers or a report to the C-level executives.

The problem with functioning in the OLAP space is that nobody knows what an enterprise has in the form of OLTP databases (you may know the amount of applications you host but how many do the individual departments host outside IT or how much Excel sheets does HR keep around?). Another problem is that in the OLTP databases you do know about, the information in it is highly unregulated, badly documented or done by a host of employees or even a third party. Then the third problem is that nobody of a seasoned DBA and/or architect and/or developer and/or business insider knows what the heck this flood of data means and how they can get a handle on it. Then the fourth problem is that even if some manager kind-of understands what he can get out of it, he'll ask for too much or too little and either it'll be unusable to his underlings or he'll constantly ask for modifications to the project that provides him with the data.

Also many, many vendors of databases and OLAP tools have promised a lot and delivered little in this space (think Oracle, MS, Peoplesoft, SAP, ...) It is thus a highly stressful job where you manage not only a shitload of data but also the expectations of sometimes a whole company and the broken pieces of your predecessors and whatever scumbag tried to sell the company his snake oil.

These types of projects are not only difficult but also require a lot of involvement on all levels. I've seen only few projects get completed successfully. In most cases the projects are small and to the point (such as combining a few data sources and getting very specific data out). In 1 other case, the project had unlimited resources (in time and money), it took them over 10 years and several millions but they have normalized practically ALL their in-house data sources over this time (such as - a customer id means this and should be implemented as such and these people should have access to it) and can basically link any data source they have to any other data source and run reports on it (which they are currently implementing with utmost ease). Yes, they paid IBM, Oracle, Peoplesoft and SAP a shitload of money too over this time but they are currently running on a combination of Oracle, MySQL and Hadoop platform which is highly customized and regulated (especially new datasources get a lot of scrutiny) but you can ask the team anything and usually within a few hours to days (depending on what you want and the required authorization) they'll have something ready that can fire you with reports whenever you want.

Step backwards, and gloat (2)

syousef (465911) | more than 2 years ago | (#38707650)

So... you're saying you've already made the switch from OLTP to OLAP and you'd like to take this opportunity to gloat about it, but you'd still like to hear from other developers what they think the prerequisites are for making such a move and what has held them back from doing all the cool stuff you're doing? Or am I missing the question?

You forgot to mention that he thinks that moving from being a code monkey to a data monkey is suppose to land him an architecture role. I would have thought his original job would see him better qualified. At best this is a step sideways but in reality it is probably a step backwards....and if he doesn't realise this it's probably just as well for all involved.

But then even slashdot's heyday ask slashdot was about clueless time wasters asking how to do their job or apply for one they weren't qualified for and had no idea about. Now that slashdot is a shadow of it's former self why would we expect the quality of these submissions to improve?

Re:Step backwards, and gloat (1)

hedwards (940851) | more than 2 years ago | (#38707974)

Since when does programming qualify one to design a house?

Re:Step backwards, and gloat (0)

Anonymous Coward | more than 2 years ago | (#38707980)

a data monkey is suppose to land him an architecture role.

The only way to enterprise architecture that I know of is to have Ivy league diploma, then you need about 5yr of experience as an analyst while you conspire to get the current architect promoted away while making you and people in your team (but not as good as you evidently) looking good.

Re:Step backwards, and gloat (1)

syousef (465911) | more than 2 years ago | (#38709460)

a data monkey is suppose to land him an architecture role.

The only way to enterprise architecture that I know of is to have Ivy league diploma, then you need about 5yr of experience as an analyst while you conspire to get the current architect promoted away while making you and people in your team (but not as good as you evidently) looking good.

Where I am competition is about as cutthroat as for most other management roles. But yes, 1 architect per 30-50 devs is sufficient, so it's quite specialized. I think job security would be the biggest issue for that kind of job...it's hard to fall back to developer because you go stale, and it's hard to find another architect role if you lose your current job.

No barrier at all (4, Insightful)

Intropy (2009018) | more than 2 years ago | (#38707630)

Moving from one specialized type of programming to a closely related type of specialized programming is pretty straightforward. Apply for such a position and you wont suffer compared with other candidates. Or, if your current employer needs something new done, do that new thing. You're not talking about a major career change here. Programming is programming. Even moving from something like standalone application GUI programming for windows in C# to back-end web service programming in C++ on Unix isn't that big a deal. If you can program, you'll pick up the new tech/language/idioms as needed and notice the striking similarity in the work you actually end up doing.

Need advice (4, Funny)

michaelmalak (91262) | more than 2 years ago | (#38707636)

I'm thinking of making the leap from C# 3.0 to C# 4.0. Does anyone have any advice?

Re:Need advice (2)

Intropy (2009018) | more than 2 years ago | (#38707710)

Go.

For it.

Re:Need advice (1)

PCM2 (4486) | more than 2 years ago | (#38708690)

Go.

For it.

Will that still work with WinRT? I'd hate to end up having to duplicate a lot of work.

Re:Need advice (1)

Surt (22457) | more than 2 years ago | (#38708416)

Well, considering the rather dire straights Microsoft is in (most of their earnings tied to dieing platform), you might want to consider Java. Databases don't seem likely to go away any time soon.

Re:Need advice (1)

Cederic (9623) | more than 2 years ago | (#38708800)

Given the way Oracle are buggering up Java, I'd be sorely tempted to find out what IBM consider their strategic language of choice.

It may indeed be Java, but I know at least one dev house is switching to Ruby, HTML5/Javascript is guaranteed income these days and C or C++ will continue to get you jobs for some time to come.

Or do what I did and get promoted. I get to draw pictures on whiteboards and let other people do the hard work :)

Re:Need advice (1)

SplashMyBandit (1543257) | more than 2 years ago | (#38708636)

This leap is the same as going off a cliff. Use Java please.

Re:Need advice (1)

julesh (229690) | more than 2 years ago | (#38708794)

I'm thinking of making the leap from C# 3.0 to C# 4.0. Does anyone have any advice?

Have you considered Java 7 as an appropriate alternative? You're much more likely to get a job as an Enterprise Architect if you make the switch.

acronym overload... (0)

Anonymous Coward | more than 2 years ago | (#38707638)

For those of us living in the "real" world of actual consumer products and wondering what these IT acronyms may even mean, here is what google found [datawarehouse4u.info]

Changed Job (1, Insightful)

1s44c (552956) | more than 2 years ago | (#38707676)

So you changed job from one thing to a highly related thing. Great, but why tell us about it?

You totally lost it on '...evolving toward becoming Enterprise Architects', Seems you have been hanging around the buzzword management types for far too long.

Re:Changed Job (0)

Anonymous Coward | more than 2 years ago | (#38707860)

The problem with buzzwords in this organization is that there isn't enough synergistic enablement between the vertical buzzword knowledge silos. If we're going to manage and exceed our clients' buzzword expectations, then we need to execute efficiently, energetically and deterministically, in as far as buzzword variability and uniqueness are concerned.

Are you on base with this?

Re:Changed Job (1)

Gothmolly (148874) | more than 2 years ago | (#38707880)

To paraphrase Syndrome:

"When everyone is an Enterprise Architect, no one will be."

I'm sick of this grade-inflation/feel-good mentality where everyone is somehow an "Architect".

Re:Changed Job (1)

sribe (304414) | more than 2 years ago | (#38707898)

You totally lost it on '...evolving toward becoming Enterprise Architects', Seems you have been hanging around the buzzword management types for far too long.

Duh. That's a necessary part of the evolution he mentioned...

Re:Changed Job (1)

heinousjay (683506) | more than 2 years ago | (#38708152)

What I really love about you guys is that you think your ignorance is a badge of honor, and you parade around this attitude while you act like some sort of "intellectual elite."

The ironies make me tingle.

Re:Changed Job (0)

Anonymous Coward | more than 2 years ago | (#38710862)

Enterprise Architects in the strictest sense are glorified gatekeepers of corporate data models. Knowing the family of schemas, e.g. star join, is essential but beyond data structures, EAs are not needed unless there is a project that ties together many not well documented data sources. The roadblock is that there are well documented fails of systems not meeting performance expectations.

My head exploded with all the acronyms (0)

Anonymous Coward | more than 2 years ago | (#38707678)

Sounded like a speech from a pointy haired boss....

Changing career? (2)

Kittenman (971447) | more than 2 years ago | (#38707726)

Reminds me of the cartoon of a dull-looking man talking to a woman at a party - text below was "You may not think it to look at me, but in my time I've been a bank clerk, bank teller and bank customer liaison officer".

Re:Changing career? (2)

sribe (304414) | more than 2 years ago | (#38707908)

Reminds me of the cartoon of a dull-looking man talking to a woman at a party - text below was "You may not think it to look at me, but in my time I've been a bank clerk, bank teller and bank customer liaison officer".

Great thing about that, you didn't even have to tell us that it was a New Yorker cartoon ;-)

Dilbert (1)

david.a.judge (1973214) | more than 2 years ago | (#38707760)

Is this a conversation from a Dilbert strip, but without the punchline?

Whats a "career"? (4, Interesting)

vlm (69642) | more than 2 years ago | (#38707774)

... the next chapter in their careers evolving toward ...

Whats a "career"? We don't have those around here in "IT related fields". I suppose if you live in silicon valley there is a chance of upward mobility, or maybe TLA .gov jobs, but for everyone else, its just luck that got us in a good spot in a downsizing economy in a downsizing company and downsizing department where they haven't axed us yet.

"Career" in general would be a more entertaining "ask /." topic. Work in the above plus the "ha ha noobs don't realize than even the concept of my job didn't exist when I was their age" and plenty of ageism whining and funny stories about nepotism and stuff like that.

Data Mining and related Tasks (0)

Anonymous Coward | more than 2 years ago | (#38707808)

To be good at such projects, you should have a decend masters degree in applied math or computer science (with a focus on basic theoretic concepts, such as logics, knowledge and meta modelling, PetriNets, graph theory, etc.).

On a side note. Next time you ask a question, don't gloat that much about how cool you are and ask your question straight. And please try to use normal language. Thanks.

Ralph Kimball and Pentaho Mondrian (4, Informative)

Dishwasha (125561) | more than 2 years ago | (#38707828)

The prerequisites to making the switch is first and most importantly having an appropriate business case for OLAP. The second prerequisite is that you've tried doing analytics in a traditional RDMS, perhaps jumped on to the NoSQL bandwagon, and you've failed at it (i.e. success for a little while but then your data eventually brings your queries down to its knees). Don't worry, failure isn't necessarily wrong, it's just you and your team needed the experience before you could make the next leap.

The risks are a knowledge jump in to an OLAP mindset from a traditional SQL mindset. Invest in you and your fellow developer's knowledge. Push back on management and sales when they want more immediate results and let them know that it will take 3-5 months to replace your current system. Do your proper technology evaluations. Learn FoodMart [microsoft.com] and Adventureworks [codeplex.com] and let them guide you down the path of good fact and dimension design. Don't snub your nose at Microsoft as they absorbed the company in the 80's that basically pioneered this stuff and made billions, but also don't take their stuff too literally as there are several products out there and some that do things better.

Read The Data Warehouse Toolkit [amazon.com] thoroughly and practice using Mondrian [pentaho.com] which is an open source Java OLAP engine that can sit on top of PostgreSQL, MySQL, and others. Find a good ETL tool rather than trying to write your own at first and don't be afraid to force your internal users to use this tool to create their facts. Don't worry if you don't get it the first time, but keep trying and keep discussing with your fellow developers as it takes a team to work out all the kinks. Later on you'll probably end up seeing how you did things wrong, but hopefully you can get most things right in the beginning.

Re:Ralph Kimball and Pentaho Mondrian (1)

jchevali (171711) | more than 2 years ago | (#38708210)

Dishwasha, I agree, that is the way forward (and I'm glad you've actually engaged with the question instead of deriding it as others have done).

Must be a slow day on Slashdot (0)

Anonymous Coward | more than 2 years ago | (#38707856)

Your lack of understanding that real programmers do everything you're talking about in the course of a year, not a career, is what's holding you back. You're never going to get to "Enterprise Architect", so you should cut over to DBA now while you can and find some very large company where you can remain hidden until you retire.

Go into management first... (0)

Anonymous Coward | more than 2 years ago | (#38707882)

Most Enterprise Architects I know started out as good programmers. They decided to become managers to boost their salary, then learned that they really sucked at managing. By then they were too high a level to return to programming so the company gave them a "lateral move" and the title of Architect. Now they go to meetings with the big shot managers and listen to vendor sales pitches; afterwards the VPs ask for their opinion in a way which leaves little doubt that their opinion had better support the decision the VPs already made based on cool buzzwords.

Re:Go into management first... (1)

Cederic (9623) | more than 2 years ago | (#38708820)

So close to the truth.

Some of us skipped the whole management piece though :)

So much fun. (0)

Anonymous Coward | more than 2 years ago | (#38707918)

Enterprise-class TLA-soup homework! Useful!

OLAP (3, Insightful)

klahnako (209184) | more than 2 years ago | (#38707948)

From my limited experience, the OLAP community is small and/or behind walled gardens, the tools are poor and closed source, and potential employers are only interested if you have experience in *their* BI tools (Pentaho, Microstrategy, Cognos, etc). Microsoft appears to be the only one trying to establish a theoretical basis for BI, but their efforts are starting to show age despite their being so much more that can be done in the field. Finally, you will be misunderstood by the majority of Rails/PHP/Web developers: The same one who think Key-Value stores and NoSQL are the height of modern technology.

That said, BI can be technically satisfying. If you get down to the SQL/MDX you will appreciate what a database can do; which allows questions to be phrased succinctly. I have seen too much code written in procedural languages (Javascript being the worst of them) that are many lines long and run atrociously slow, that can be restated in SQL (or MDX) simply, and run a 1000x faster. I love that fact there are no loops!

From a business perspective, you have much more exposure to management and other departments: You will have improved visibility in the company, and your worth will be inflated - as you will be the one that satisfies management's appetite for more information to help make decisions.

Different Disciplines (2)

afabbro (33948) | more than 2 years ago | (#38707958)

Your previous experience as a developer/DBA is largely irrelevant. Data analysis is a completely different discipline (and depending on tools, may even be a different language than SQL).

Basic building cubes is something you could learn in 5 minutes. Advanced stats, how to look for patterns, what to look for, etc. is much more involved. Most of the people I know who are good data analysts have advanced degrees in mathematics.

BTW, why do you think the career path is dev -> data analyst -> enterprise architect? Completely irrelevant. Plenty of data analysts who couldn't write code. Plenty of EAs who couldn't construct an OLAP cube - they're focused on infrastructure, apps, etc. EA really has nothing to do with data analysis, other than designing systems to support it. In many companies, EA is not some priesthood at the top of the food chain - often they're a virtual team made of different disciplines.

To the OP (1)

vikingpower (768921) | more than 2 years ago | (#38708284)

Buddy, listen. Moving from OLTP to OLAP should be a minor, if not a micro move for anyone acquainted with database design. Don't waste our time and bytes with it.

As for becoming an "architect", this is how I became one: I took all the ( little ) experience I had, I designed some stuff, made sure the code monkeys could and would actually code it ( me knowing their life as I had always been one ), got some $$$ incentives for them so they would build what I had thought up. I defended it in hard fighting with the management. Then the stuff went into production. It took balls, hard work and some luck - and, yes, some politics. "The rest is ashes and dust", as Russell Crowe has it in "Gladiator".

Be prepared to slow down (1)

codgur (1518013) | more than 2 years ago | (#38708354)

I made a similar transition and my furious pace and insistence on sub 300 millisecond response times for any query, or web page load was foreign to those in the OLAP world. I would submit a query to our OLAP (I was doing pricing optimization—cool enough) which took over 60 minutes to get a useable dataset. Most of the people I worked with were very nice and extremely bright though they existed on a very relaxed pace; one I just couldn’t adjust to. I had such a tough time of it I moved back to the OLTP world where someone would listen to my insistence, enjoy my furious pace and manic persona and I couldn’t be happier. Do give it a try though. It is very rewarding to be able to analyze all a company’s data and write one query that will help generate more profit. I do wish you well and just remember to slow down.

Re:Be prepared to slow down (0)

Anonymous Coward | more than 2 years ago | (#38710522)

As good Marxists we should all keep in mind that OLAP may be evil because it can help corporations generate more profit.

Therefore we must ensure that it's only used for good: Helping government make sense of the monitoring data it's collected on the citizenry. In the pursuit of progressing towards a more orderly and equitable and perfected society, of course.

Read this book.. (0)

Anonymous Coward | more than 2 years ago | (#38708610)

Read the book, Star Schema by Robinson. Also, you can do both OLTP and OLAP - really, they complement one another.

What I've seen... (0)

Anonymous Coward | more than 2 years ago | (#38708772)

As a system administrator, I've supported OLAP processing for a good part of my career, although I'm not doing that work currently. What I've seen is that originally a lot of the data warehousing complex was used just to run the database with it's billions of rows and many terabytes of data, running something like Redbrick or maybe Sybase IQ. Then we started to spend more time handing "staging" (ETL) workloads, with tools like Ab Initio, SyncSort and N-sort. Every year, the amount of data increased, the number of details increased. We were constantly fighting to get the reports done in our 24 hour window. If you get behind, you may be catching up for awhile.

These days, it's all about using Hadoop to prepare the data for load into a database, either Oracle 11g RAC using ASM, or perhaps a Netezza box. I see some of the column stores like Vertica becoming important for some workloads in the near future.

As a result, I'd say that people wanting to move into data warehousing should learn Java, Hadoop, Oracle RAC. Keep an eye on HBase, Hive, and the NoSQL solutions for the future.

One thing that I'll mention that I've found very useful is that throughout the time I was working on this stuff, perl was very handy. I imagine other scripting languages would be useful, too, but in my case perl provided the glue that made everything work.

Useless (0)

Anonymous Coward | more than 2 years ago | (#38709108)

OP asked a reasonable question and most responses were either wisecracks or otherwise simply ignorant and/or useless. Good thing I only come here for the most part to see what the tech headlines are; otherwise, I'd be disappointed when I read the usual half-backed, leftist, ignorant MS-bashing rants that pass for commentary on this site.

Re:Useless (0)

Anonymous Coward | more than 2 years ago | (#38710282)

Agreed. It's pathetic to realize that:

1. There are a bunch of poseurs posting on /. who like to think they're all superior and cool and super elite technical and yet don't know what OLAP and OLTP are. (Thank goodness the submission didn't talk about HOLAP and ROLAP.) Hint - if you're in the IT industry more than, say, five years and you seriously haven't heard of OLAP, then you're not a professional, you're just playing games.

2. Considering the massive amount of insults shoved at MBAs on this site, it's particularly ironic to know that a halfway decent MBA course probably includes an introduction to OLAP because it's so fucking important.

SQL vs MDX, DMX (0)

Anonymous Coward | more than 2 years ago | (#38710254)

Once you go beyond data warehouses and ETL, the programming language skills required for OLTP and OLAP are different
MDX for multidimensional cube queries - can get a little hairy
DMX for data mining models - a bit limited in terms of extensibility
Correct cube design is also an issue - use case specific dimensions & measures; storage price / perf ratio

been there (0)

Anonymous Coward | more than 2 years ago | (#38711044)

I started my career with OLAP & BI, been at it for a decade or so now. Also doing OLTP like everyone else.

Not sure what to say, it's just bits and pieces, and you need to process those so that you can meet requirements and things like that.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?