Welcome!

Agile Computing Authors: Pat Romanski, Zakia Bouachraoui, Elizabeth White, William Schmarzo, Liz McMillan

Related Topics: @CloudExpo, Agile Computing

@CloudExpo: Article

Cloud Computing: Automation's Great - But I Trust Humans More

Opinion: Cloud Computing vs Humans

Brendan Cooper's Blog

We need services like Feedburner and Yahoo Pipes to provide the service they say they will. I know they’re not bound by the kind of service level agreements that would be in place if we were actually paying them, but they surely have to operate within the bounds of, well, operability. Because if they don’t, someone else will.

Repeatedly I’ve found myself being let down by the latest and greatest RSS tools.

I’m not really sure how I can get around this problem, other than by being constantly vigilant - to the extent that it’s almost easier to forget RSS altogether and just monitor ‘by hand;’.

This is how it should work

I’m currently working on the best way to use the various RSS tools out there to create a flexible yet powerful monitoring system. To my mind a good solution is:

* Yahoo Pipes for processing. Yahoo Pipes is an RSS mash-up service, so you can do stuff like take feeds and bring them together, split them apart and filter them, all through a nice graphical interface. It is so flexible and powerful, and you can create really nice modular pipes that slot together like Lego. Want to search for blogs? Insert your ‘blog search engine’ pipe. News? Same. Microblogging? No problem. One of these days I’m going to put a quick tutorial about Pipes on this blog. But not for a while yet. I explain why below.

* Feedburner for future-proofing. Feedburner is an RSS ‘add-on’ service, enabling you to add titles and descriptions to feed but, for my money, the most important feature of which is the feed renaming service. So if you have a feed with the URL http://x.y.com/feeds/asdhJAH72jjaaSS99.xml, just plug that into Feedburner at one end, tell it you want it to be called ‘My lovely feed’ instead, and from then on it also has the URL http://x.y.com/mylovelyfeed. So whatever RSS feed is coming in, it retains the same address. This means that, if you have to use different feeds coming in, you don’t then have to scratch around looking for whatever you had those feeds plugged into. If you know they’re going into Feedburner then you just have to change it there. Everything else stays the same.

* Google Reader for archive and analysis. Google Reader is an online RSS aggregator, so all you need is a Google account and you can use Google Reader’s very powerful features with no installs or upgrades needed. So, you’ve got a cool RSS feed created from Yahoo Pipes, going into Feedburner and retaining the same lovely name. You’re now ready to plug that lovely feed into any other RSS-enabled tool. The next thing you want to do is analyse it, so plug it into Google Reader and suddenly you can filter for ad-hoc queries, star or share items, go through archives, even produce web pages for clients and extra feeds.

* Netvibes for display. Netvibes is an online RSS aggregator too, but while Google Reader’s good if you like lists, sometimes people like columns. So take your lovely RSS feed and this time use Netvibes to create a ‘front end’ for your feeds. So, you monitor the Google Reader stuff, while the client gets to see a really neat dashboard type display. You can add charts and all sorts of bells and whistles.

There just one problem: it doesn’t

I know this can work. I’ve seen it work. But there are frustrations along the way and recently I’m starting to wonder whether these services can be relied upon to work.

A couple of weeks ago I noticed that some critical Feedburner feeds had ‘died’. I was relying on them for data to come through for some important monitoring work. On further inspection I noticed one of the feeds had gone above the 512KB limit for Feedburner, but, while annoying, that didn’t explain the other problems I was having. Other people were commenting on Twitter about similar problems, and about the same time I noticed my Feedburner-enabled subscriptions had halved.

The solution was to bypass Feedburner altogether and just use the Yahoo Pipes addresses instead. But this was far from ideal. I want to use Feedburner for control over the address. I want to feel I can rely on it.

But the real culprit in all this is turning out to be Yahoo Pipes. I have invested considerable time and effort into getting to know it. I’ve got a system that builds queries from keywords, goes out to just about every RSS-enabled social media source I can find, grabs those feeds, filters for them, appends information to the titles and spits them out in virtually any configuration needed. I’ve tested it all, and I know it works.

But about two weeks ago I noticed Yahoo Pipes getting sluggish. It didn’t matter what I was using to access it - my PC at home, my laptop at work, IE, Firefox, whatever.

And this weekend, I can’t do anything with it. I need to add some tweaks to the system, but it either times out, or refuses to save my changes.

I mean, as I said earlier, I’d love to pass on some of what I’ve learned on this blog. But I cannot. Even when the system is working it’s just too slow. I find I’m wandering off to stroke the cat or do the crossword while Yahoo Pipes churns away.

So again, I have to ask: can I rely on it?

I can’t see the silver lining for the cloud

This is, of course, a criticism of cloud computing. While I absolutely love the idea of harnessing the power afforded me by Yahoo’s server farms to do weird and wonderful things with RSS, I hate, detest and loathe the notion that I’m totally dependent on them to be able to do so.

If, as has been happening for the past few months, I continue to creep into the cloud, I know that one day I’ll have really seriously important stuff in, say, a spreadsheet on Google Docs, that I cannot access when it’s critical that I can access it. Or I’ll get into trouble with a client because they’ll blame me for not making sure their RSS feeds are working properly.

Is the answer that I just don’t put all my trust in these services? Do I keep local versions of docs, just in case? In which case, what do I do for RSS monitoring? I mean, can I pay someone money to give me a better service? Is that the real issue here?

Perhaps the Luddites are right

So, to get back to my original point: at what point do I totally lose faith in these services?

I’ve spent enough time testing my systems to know that they work. The problem is, the services themselves don’t seem to work properly.

So do I monitor constantly and vigilantly to make sure everything is tickety-boo? Do I just hope that, come the day I’m dependent on Yahoo Pipes to work, and it doesn’t, I can quickly think of a workaround as I did the other day?

Or do I eventually decide that actually, it’s more reliable and in the long run more cost-effective simply to monitor individual blogs by visiting them on a daily basis? I mean, there’s something to be said for this. I would certainly get to know those bloggers more intimately. But this solution just doesn’t scale up. It’s not workable.

No. We need services like Feedburner and Yahoo Pipes to provide the service they say they will. I know they’re not bound by the kind of service level agreements that would be in place if we were actually paying them, but they surely have to operate within the bounds of, well, operability.

Because if they don’t, someone else will. I’ve already been checking out Microsoft’s Popfly mashup creator today to see if it can do what Yahoo Pipes should. And it’s already looking promising. We’ll see.

 

[This appeared originally here and is republished in full with the kind permission of the author, who retains copyright.]

More Stories By Brendan Cooper

Brendan Cooper is a Digital Senior Account Manager at Fleishman-Hillard in London, UK. He has spent the past 20 years laboring at various communications coalfaces. A graduate of the Information Systems Institute at Salford University, in the UK, he was a founder member of sharepages.com and, from that, Knowledge Technology Solutions PLC.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


IoT & Smart Cities Stories
Dion Hinchcliffe is an internationally recognized digital expert, bestselling book author, frequent keynote speaker, analyst, futurist, and transformation expert based in Washington, DC. He is currently Chief Strategy Officer at the industry-leading digital strategy and online community solutions firm, 7Summits.
Digital Transformation is much more than a buzzword. The radical shift to digital mechanisms for almost every process is evident across all industries and verticals. This is often especially true in financial services, where the legacy environment is many times unable to keep up with the rapidly shifting demands of the consumer. The constant pressure to provide complete, omnichannel delivery of customer-facing solutions to meet both regulatory and customer demands is putting enormous pressure on...
IoT is rapidly becoming mainstream as more and more investments are made into the platforms and technology. As this movement continues to expand and gain momentum it creates a massive wall of noise that can be difficult to sift through. Unfortunately, this inevitably makes IoT less approachable for people to get started with and can hamper efforts to integrate this key technology into your own portfolio. There are so many connected products already in place today with many hundreds more on the h...
The standardization of container runtimes and images has sparked the creation of an almost overwhelming number of new open source projects that build on and otherwise work with these specifications. Of course, there's Kubernetes, which orchestrates and manages collections of containers. It was one of the first and best-known examples of projects that make containers truly useful for production use. However, more recently, the container ecosystem has truly exploded. A service mesh like Istio addr...
Digital Transformation: Preparing Cloud & IoT Security for the Age of Artificial Intelligence. As automation and artificial intelligence (AI) power solution development and delivery, many businesses need to build backend cloud capabilities. Well-poised organizations, marketing smart devices with AI and BlockChain capabilities prepare to refine compliance and regulatory capabilities in 2018. Volumes of health, financial, technical and privacy data, along with tightening compliance requirements by...
Charles Araujo is an industry analyst, internationally recognized authority on the Digital Enterprise and author of The Quantum Age of IT: Why Everything You Know About IT is About to Change. As Principal Analyst with Intellyx, he writes, speaks and advises organizations on how to navigate through this time of disruption. He is also the founder of The Institute for Digital Transformation and a sought after keynote speaker. He has been a regular contributor to both InformationWeek and CIO Insight...
Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settlement products to hedge funds and investment banks. After, he co-founded a revenue cycle management company where he learned about Bitcoin and eventually Ethereal. Andrew's role at ConsenSys Enterprise is a mul...
To Really Work for Enterprises, MultiCloud Adoption Requires Far Better and Inclusive Cloud Monitoring and Cost Management … But How? Overwhelmingly, even as enterprises have adopted cloud computing and are expanding to multi-cloud computing, IT leaders remain concerned about how to monitor, manage and control costs across hybrid and multi-cloud deployments. It’s clear that traditional IT monitoring and management approaches, designed after all for on-premises data centers, are falling short in ...
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
Dynatrace is an application performance management software company with products for the information technology departments and digital business owners of medium and large businesses. Building the Future of Monitoring with Artificial Intelligence. Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more busine...