Low Cost Webcast Optimizations? 108
ChunKing asks: "I work for a small community broadcasting organization, and we operate a limited streaming media facility for a number of not-for-profit webcasters. It has always been an issue to optimize our streaming media infrastructure to most benefit our users. We operate a small cluster of servers from a data center with good connectivity and a highly-rated ISP, who will occasionally allow us to burst to unlimited bandwidth. For big webcasts, we will load balance the stream over a number of servers using round robin DNS. However, we still get problems with stream buffering and network drop-outs, particularly with streaming video. We cannot afford a network of edge delivery servers like Akamai, so in what ways can we further optimize our streaming media capacity to better produce smooth webcasts?"
Torrent (Score:3, Interesting)
I wonder how well a method such as bittorrent adapted to streaming data would do - i.e. decentralized streaming.
The first few viewers receive the stream directly from the source, while acting as mirrors for subsequent viewers. It would dramatically cut down on the server load & overhead for the broadcasters.
Re:Torrent (Score:4, Informative)
Re:Torrent (Score:2)
Re:Torrent (Score:3, Insightful)
But, again, a really, real
Re:Torrent (Score:2)
Seems to have worked well for Skype.
Re:Torrent (Score:2)
Re:Torrent (Score:2, Informative)
You can find their informal wiki page at http://wiki.xiph.org/IceShare [xiph.org]
Re:Torrent (Score:3, Informative)
Kinda like this? [dijjer.org]
Re:Torrent (Score:2, Informative)
Re:Torrent (Score:2)
Re:Torrent (Score:1)
Low Upstream (Score:2)
I do not see sufficient bandwidth for distributed streaming in the near future here.
Re:Low Upstream (Score:1)
You're probably going to want to do forward error correction, dedicationg 5% or so of the stream to error correction, rather than wasting the bandwidth the other way with acks.
OTOH, in some cases, such as radio, 128K is just fine, and is a lot better than 44K that some stations webcast in noe.
Re:Torrent (Score:2, Informative)
Re:Torrent (Score:1)
Unlimited Bandwidth (Score:5, Informative)
"who will occasionally allow us to burst to unlimited bandwidth"
Well, unless your provider has "unlimited bandwidth" themselves, you're basing your webcast on a lie. Pretty much no one has unlimited bandwidth; even a couple hundred broadband streams can saturate an OC-3 connection.
What you need to do is plan your webcasts a little better and ask a bunch of questions: What's the real bandwidth your ISP can provide (with redundancy)? What's the buffer size that your client apps are using? (Settable in some clients, like Flash.) Does your ISP (who promised you unlimited bandwidth) even know what the hell they're doing?
Without going to a dedicated CDN like Akamai, it's still pretty easy to design a distributed service network with colocated servers scattered across the country. You might want to consider finding someone who knows this kind of thing and paying them a few bucks to fix your problems...
Re:Unlimited Bandwidth (Score:4, Funny)
Re:Unlimited Bandwidth (Score:2)
Re:Unlimited Bandwidth (Score:2)
Marginally more original. I'd at least facilitate cutting-edge paradigms to visualize tracking users and facilitate clicks-and-mortar deliverables deployment to leading-edge consumers
-jk [robietherobot.com]Re:Unlimited Bandwidth (Score:2, Funny)
Yeah, well *I'm* on Internet III (Score:1)
-Eric
Re:Unlimited Bandwidth (Score:4, Interesting)
After all, even if they scale up with colocation and distributing their webcast to other servers, if they can handle [x] users but only get [x/2], with a burst of x+2 once every 6 months, it seems like they should focus more on what they can do to reasonably control dropouts among users who are already connected, vs the rare scaleup to accomodate a large number of users for a short period of time.
Of course, it might also prove out that by having more solid streams, they'll have more users, and will inherently get more users, which would lead to an escalation of costs. Might be screwed either way.
Re:Unlimited Bandwidth (Score:2)
Re:Unlimited Bandwidth (Score:1)
Forget webcasts... (Score:2, Interesting)
streaming is so 1997 (Score:5, Informative)
Re:streaming is so 1997 (Score:5, Informative)
People like the simplicity of just clicking a link and the video poping up and streaming in media player, no download overhead eather.
Re:streaming is so 1997 (Score:3, Interesting)
Torrenting is the new downloading, if people want to get the latest and greatest on the net then they'll just have to learn how it works. I'm sure that it will just eventually be configured automatically as well though.
Re:streaming is so 1997 (Score:2)
87% IE, 12% Mozilla after four years. Lesson learned: people don't want the latest and greatest.
Re:streaming is so 1997 (Score:2)
Plus more and more people are behind firewalls and routers
That's a very good point. Along those lines, many are behind routers/firewalls that actively block P2P and torrent traffic as a matter of policy. (e.g., at universities, businesses, etc.) So, depending upon the target audience, relying solely upon bittorrent could cut out a major group. -- Paul
Re:streaming is so 1997 (Score:2)
Re:streaming is so 1997 (Score:2)
Another really good port btw is 1863, the MSN Messenger service uses this port, it's above the 1024 service level (so easily setup for use by IRC daemons on shell accounts for example), allowed through most firewalls, and, like 443, it's already SSL so won't raise
Re:streaming is so 1997 (Score:2)
Re:streaming is so 1997 (Score:3, Informative)
Re:streaming is so 1997 (Score:2)
Re:streaming is so 1997 (Score:2)
You must be thinking of Javascript's XMLHttpRequest object.
Re:streaming is so 1997 (Score:1)
How can you say this when the average person will download a special media player just to play a certain site's files (name your DRM), or download special applications just to have certain mouse cursors (CometCursor), or download special applications to get a weather icon on their taskbar, etc etc. These are all targetted at the average user. Als
Re:streaming is so 1997 (Score:5, Informative)
Unfortunately, though, a CDN-style infrastructure is still the optimal way to go. There are generally resellers of the CDN technologies that can give you reduced rates on the same service. I know our particular model allows for easy scalability with tools that allow you to control your bandwidth usage. </pitch type="shameless">
Re:streaming is so 1997 (Score:3, Informative)
Keep up the good work!
Re:streaming is so 1997 (Score:1)
Re:streaming is so 1997 (Score:1, Interesting)
For instance, if you produce a video that uses a song from Britney Spears (not that anyone would ever do that, but play along), the licenses required to allow people to physically download copies of that song to their hard drives are different and usually vastly more expensive than releasing the video as a live stream or as part of an on-demand system. The primary question of this article was maintaining fiscal responsibility, not opening the
Do they *have* to stream? (Score:4, Interesting)
Re:Do they *have* to stream? (Score:2)
Re:Do they *have* to stream? (Score:1, Informative)
Re:Do they *have* to stream? (Score:2)
3 steps to improve streaming (Score:3, Informative)
Second, if it absolutely has to work, you can rent services from Akamai, who have a huge and under-used network for exactly this service. Let them deal with distributing the load from one feed to thousands of servers worldwide: they think it will make them lots of money. They will play more clever DNS games than merely round-robining.
T
Not more bandwidth but.. (Score:1)
One word. (Score:3, Informative)
Re:One word. (Score:2)
You are correct (Score:3, Informative)
Seriously, because ISPs generally do not support multicast (even though their hardware does, it's just disabled, it requires 5 seconds to turn on, and is likely native downstream of them anyway), it is better to do this the way NASA Select did with their TV broadcasts in the days of CU-SeeMe. (Great program, died an ugly death, and is now a zombie in the hands of a corporation.)
What NASA did was to multicast for anyone who could
Re:You are correct (Score:3, Informative)
Multicast is one of those technologies which was designed quite well and makes perfect sense in theory, but is seldom useful.
If this guy is broadcasting to one person in Topeka, one in Taipei and one in Tennessee, does he need multicast? He doesn't. Should every router on the Internet participate in the creation of PIM trees or allow the inclusion of hundreds of thousands of /32 routes under the 224 prefix? They shouldn't. So is multicast useful for streaming au
You are in error at line 10 (Score:3, Insightful)
Think again. Three copies means you chew up three times the bandwidth to get the same file out. If you multicast, you can: (a) transmit to all three the same information at the bandwidth of a single person, (b) transmit to all three three times the information at no extra bandwidth, (c) anywhere in between.
The first option basically means you can run a broadcast station and be doing ot
Re:You are in error at line 10 (Score:2)
So every core router on the Internet should passively accept multicast JOINs and maintain state for every single multicast session going on across the entire Internet? I don't think so. Do you have any idea how vulnerable that would make your core? There's a reason the MBONE is routed discretely.
Turning on multicast so you can process OSPF HELLOs on 224.0.0.5 doesn't mean you're multicast routing. Your argument that
CUseemee (Score:2)
Re:CUseemee (Score:2)
A bunch of CU-SeeMe enthusiasts, meanwhile, produced a full colour version of the free program. I don't believe the colour interoperated with the White Pine
Yes. (Score:2)
If you want your box to act as a "junction" between multicast islands, you'll want a software multicast router - pimd is good, I'm not sure if Zebra of Xorp support multicasting but they really should at this p
Re:One word. (Score:1)
Re:One word. (Score:2, Informative)
bottleneck? (Score:5, Informative)
You mention a lot of factors, but you haven't said what area you're having the most problem with. Have you at least managed to identify what the actual bottleneck is? It could be bandwidth at your servers, or between your ISP and some peer (through which most of your traffic is going), or it could be CPU load on your servers, or it could be disk I/O limits on your servers, or it could be lack of memory on your servers (causing them to thrash), or it could be uneven load balancing to DNS-based load balancing being somewhat random, or it could be that you've reached the maximum capacity (in terms of hosts per response record in DNS) with DNS load balancing and you need to add another layer of load balancing.
The first step is going to be to check into all those areas and identify where the failure is currently happening. Once you know that, the solution will be more obvious. You might just need to add striping for your disks (to increase throughput by brute force) or RAM (to be able to cache all active streams) if the problem is that your disks can't keep up. But, if your ISP's connection to a major peer isn't sufficient, then you will need to do something much more involved (and will probably include gathering data during an actual event where the streaming can't keep up in order to prove that the ISP is the problem).
Akamai and other Content Delivery Networks (Score:4, Informative)
But to return to your original question - since you're dealing with not-for-profits, clearly the answer is a live streaming P2P "edge network". Let me know when you release that code
If you don't have the development resources for that, the best solution is probably to scale down your investment in your current colo and pop a few servers in a different geographic locale, a kind of DIY edge network. With 3 or 4 colos (say, Europe, US-East, US-West), and smart determination of which host to route to using Geo::IP or the like, you could probably save a lot of traffic and latency.
Outsource it to Google video (Score:5, Interesting)
Upload video to Google [google.com] video or YouTube [youtube.com].
Use their bandwidth to stream your video.
Use their API to embed the video on your website.
You lose control over the distribution of the content, but you save a lot of money in the process. The other option is Bit torrent.
Also, have you looked into DigitalBicycle [digitalbicycle.org]? They are working on a PeerCasting system for Using Bit Torrent, RSS, and web community software
Start your own "Flickr of Video" (Score:4, Informative)
Second on YouTube and Google
I agree there is something to be said for "out sourcing" your video to a company like Google Video or YouTube, but if you don't mind I would like to throw my little Start-Up (Vidiac.com) into the ring since we've been doing free video hosting longer than either of those two without all the Web 2.0 Hype. Of course the difference with us is we give you a video hosting portal you can run on your website (Like video.YOURDOMAIN.com), so with us you can start your own "Flickr of Video" either free (ad supported) or pay-per stream. Sorry about the self promotion there.
Regarding your Dilemma
Here is my input on the question at hand based on my own experience (I'm streaming just over a million streams a day to 2.5 Million Unique visitors a month). First, as always, start with the needs of the users.
1.) Are they sophisticated? How would they react to a Torrent? From my experience, if someone has a torrent client installed then they're fine with it, but if they don't you have a large "fear factor" to overcome convincing a user to install a program to watch your video.
2.) Do they want to click and Watch, (stream), or do they want to download? If you produce content like a video podcast, then I highly recommend giving your users a download option over streaming. If you run a portal-site, like Big-Boys.com that aggregates content then you're better off streaming as you want to be viewed as more than an FTP site.
3.) How big is your Audience? This is probably the single biggest factor for capacity planning. If they all hit your site at once you need a ton of bandwidth.
4.) Is this a scheduled broadcast (like a weekly show), or is this content that will be downloaded over time? Scheduled events can cause massive spikes. 10,000 users all at once is harder on a server than a Million users spread out over a month.
5.) What video quality will your users tolerate? What Software can they use to view it? Windows Media? Quicktime? Flash? Size? Bit Rate? If your users are watching these videos on an iPod they'll accept a lower quality than if they're viewing it on their Plasma HDTV.
Based on the above you can get a firm handle on your
1.) Delivery Method (Direct Download, stream, Torrent)
2.) Video Size (Format, Quality, etc)
3.) Pipe Size (How much do you need to stream at once)
3.) Bandwidth Utilization profile (spikes and valley vs. flat)
Compare the above to your current solution. If your users are buffering, is it because...
Your upload pipe is saturated? Upgrade to a larger pipe (100Mbps is about the minimum if you have more than 10 people hitting your video at the same time)
Is your Disk I/O saturated? SATA and IDE are fine for most sites, but if you have 50 viewers requesting 30 different videos you need some strong I/O. we were burning through SATA drives on a weekly basis before switching to a SCSI SAN.
Does your ISP have proper Network access to get to your end users? Consider pulling in a pipe from an appropriate network. (AT&T strongly peers with Cox cable for instance, MCI has fantastic AOL connectivity).
Anyhow, I hate to answer a question with more questions, but there are so many ways to deliver video, there is no one silver bullet solution. You just have to do what works for your users and company. From experience though capacity planning for an online video application takes a lot of work, but once you wrap your head around the bandwidth issue, it gets easy. Good Luck!
-Adam
Re:Outsource it to Google video (Score:2)
Just got with torrents.
Use decent modern compression for streams (Score:3, Interesting)
For video? I'd recommend XviD [xvid.org], and not just because I use it almost exclusively, but it creates a compliant MPEG-4 video stream (compresses nicely), which will hopefully be streamable by any player that supports MPEG-4 video.
Re:Use decent modern compression for streams (Score:3, Interesting)
The difference IS noticeable the lower down the bitrate you go.
So what, you'll need more resources to encode the video but its worth it.
Re:Use decent modern compression for streams (Score:1)
Re:Use decent modern compression for streams (Score:1)
so many bad answers.. (Score:2, Insightful)
Clean, Secure Bandwidth (Score:2)
How small is that cluster? and what value is unlimited and burst? You don't say which brand of streaming you're running. It wouldn't be meaningful for me to extrapolate from my single QuickTime X-Serve experience, but I have saturated a 100mb/s link between my site and a test site with multiple streams, and no sign of stress on the box. The
bitrate (Score:2)
Re:bitrate (Score:2)
In any case, for video watched on a TV vs. a computer monitor, I find that even 320x240 is very watchable (roughly equivalent to VHS videotape, I think) if the frame rate remain
What's the Problem Precisely (Score:2)
It's probably that your bandwith is being maxed out... but you'll want to hire someone that knows what they're doing. That'd be the place I'd start, were I in your shoes.
Ourmedia.org (Score:3, Informative)
Investigate Ultravox (Score:5, Interesting)
http://ultravox.aol.com/ [aol.com]
What's your input like? (Score:2)
More items for your checklist -
It's not the servers... (Score:2, Informative)
Sounds like bandwidth (Score:5, Informative)
Also, if it is possible, try to expirament around with variable bit-rate streams for video and/or audio. That can come in handy if your target audience spreads from AOL dial-up in the middle of nowhere to your urban 8 Mbps+ broadband connections.
In streaming, the big three big things that you have to worry about are the encoder's connectivity (to push the stream to the distribution point), the encoder's power (to make sure it can really compress all that video without overheating or dropping frames), and the distribution point's connectivity. Distribution servers don't use much bandwidth as all they do is simply regurgitate the data back out to clients without having to do any processing other than logging.
I've been doing streaming for about 5-6 years now. I've done everything from web cam streaming to large convention streaming that put out 300+ Mbps of connectivity. By no means am I an expert (I've done streaming--it's not what I do), but everything above is my experience with it. Most of all, just stick with what you know and are comfortable with. Your client base will determine most things. Personally (even being a BSD guy), my preferred streaming setup is using Windows Media Encoder at the source, Windows Media Server 2003 at the distribution point, and Windows Media Player (or anything else that can play Windows Media v9) at the end-user. In my experience, it has been rock-solid and can be deployed quickly. If you're already running Windows 2000/XP/2003, all the software mentioned above is provided free-of-charge.
Hope this helps.
Think TiVo, not Live TV. (Score:2)
Set up a video podcast. You may host it all that way, but it's not designed around having everyone sucking bandwidth at the same time. You'll want a delayed download option anyway so those who can't sit in front of the screen whe
How about something free? (Score:2)
Not having done anything like it before, it was not TOO difficult for me to set up.
podcasting best bet (Score:1)
Wow (Score:1)
Scrap the Junky Client (Score:1)
you need to pick a solid client, and the only thing that does a decent job is (irk, windows media) or flash, but flash eats bandwidth for lunch.
Re:Scrap the Junky Client (Score:2)
Look at what Pr0n is doing (Score:4, Interesting)
Carnegie Mellon University's P2P Video Streaming (Score:2, Informative)
Contact the folks at Carnegie Mellon University's "End System Multicast" gang. Its a P2P video streaming sysytem!
It's at http://esm.cs.cmu.edu/ [cmu.edu]
LX Systems is able to do what you want via p2p (Score:1)
http://www.lxsystems.com/index.html [lxsystems.com]
lbnamed-loadbalance improvement to round robin (Score:1)
Works great, we use it for application login servers, proxy pac file sites, citrix servers, etc. DNS will return IP address of least loaded machine. simple easy and definately cost effective.
Freecast, Peercast (Score:1)
http://www.peercast.org/ [peercast.org]
and
http://www.freecast.org/ [freecast.org] (java based)
yomguy
Don't know if they're cheap... (Score:2)