dslreports logo
 
    All Forums Hot Topics Gallery
spc
Search similar:


uniqs
9681
Stewart
join:2005-07-13

2 edits

8 recommendations

Stewart

Member

Almost free PBX in the Google Cloud

I've recently have been experimenting with Google Cloud Platform and am quite impressed. Their scheduling latency performance of their shared-core VPS offerings is far superior to the cheap stuff on LowEndBox and LowEndStock (IMO unsuitable for production PBXes) and at least equals that of RentPBX, Linode, DigitalOcean, Rackspace, RamNode, etc.

The platform includes hardware firewall protection, snapshots, ease of creating / moving / destroying servers on demand, as well as many other management features.

In addition to their one-year free trial with $300 credit applicable to most services, many services offer truly free usage (not a trial) within certain limits. For details, see »cloud.google.com/free/ . These limits are sufficient to run a small PBX indefinitely, except for egress bandwidth priced at $0.12/GB after the first GB/mo. For example, if talking 2000 minutes monthly, once your free year is up you'll be billed ~$0.25/mo.

Here is an easy way to try FreePBX on GCP: I took the instructions at »wiki.freepbx.org/display ··· CentOS+7 and assembled them into a script, adding modifications for GCP and automating all interactive steps.

Once you've signed up, go to »console.cloud.google.com .
Select Compute Engine -> VM instances.
Click Create Instance.
Choose a name, select zone; free options are us-east1 (South Carolina), us-central1 (Iowa) or us-west1 (Oregon).
If you're not sure, check at »www.gcping.com/ (test takes about 30 seconds to run, actual ping time is ~12 ms less than HTTP time reported).
(Leave default Machine type for faster build -- we will change to a free one later.)
Change Boot disk to CentOS 7, press Select.
Under Firewall, click Allow HTTP traffic. Click Create.
On the VM Instances page, under Connect, click SSH (need to click twice if blocking popups). A console window will open (takes ~30 seconds).
Paste the code below into the console window.
sudo -i
curl http://x-sc.com/fpbx14.sh > fpbx14.sh
chmod +x fpbx14.sh
./fpbx14.sh
  
 
When the script is done (about 20 minutes), "You have successfully installed FreePBX" should appear in green. If a minute goes by with no activity, it has probably stopped on an error.
Open the external IP address in your browser.
Set up admin account as requested, log in, set timezone, etc. Apply Config.
With luck, there will be no errors shown in the System Overview.

Go to Settings -> Asterisk SIP Settings. Click Detect Network Settings. The External Address field should populate. Click Submit then Apply Config.
In the cloud Console tab (not the SSH console window) go to VPC network -> Firewall rules, click Create Firewall Rule.
Give it a name e.g. sip. Under Target tags, type a tag name, e.g. sip.
Under Specified protocols and ports enter "udp:5060; udp:5160; udp:10000-20000" (without the quotes and assuming default FreePBX settings). Under Source IP ranges, enter 0.0.0.0/0 (or a restricted range if desired), then click Save.
Go to VPC network -> External IP addresses. For your instance, change the Type from Ephemeral to Static. Give it a name, e.g. mypbx. Click RESERVE.
Go back to Compute Engine -> VM instances. Select your instance, click EDIT. Under Network tags add sip or whatever tag you chose above. Click Save.
Reboot (needed after network settings changes, also to confirm that automatic startups are working). You can reconnect to SSH server after one minute.
In the FreePBX UI, configure an extension. Set up a VoIP device or app to use it.
Call *43, confirm that the echo test works and latency and quality are ok.
In the FreePBX UI, configure a trunk and an outbound route.
Confirm that you can make a call and that latency and quality are ok.
Restrict access to the FreePBX GUI. For example, remove the http-server tag and add a rule that only grants access to your PC.
Close the SSH console window.
In the Console, stop your instance. Go to Snapshots. Click CREATE SNAPSHOT.
Give your snapshot a suitable name, e.g. as-built. Under Source disk, select your instance's boot disk. Click Create.
Edit your instance, change Machine type to micro (will become free if in qualifying zone), click Save.
Start your instance up again.
In the FreePBX UI, go to Module Admin. Click Check Online, select desired Repositories, install modules as desired.
Configure PBX for your application.
Take another snapshot.
Enjoy.

I'm aware of two problems with this build:

1. It can't send mail (such as voicemail notifications), because Google blocks outgoing ports 25, 465 and 587. My mail provider, Fastmail, has a proxy that avoids this issue. I'll try some of Google's suggested general purpose workarounds and will post an update if successful.

2. It crashes after configuring a Google Voice account. An apparently good motif.conf is created but Asterisk segfaults after reading it. I had the same problem with a FreePBX 13 / Asterisk 13 build. If you just install the FreePBX module without an account (creating dummy motif.conf and xmpp.conf files) and reboot, Asterisk will read those and start running the respective modules. I have no idea what may be wrong and would appreciate it if a GV expert could take a look. Create a snapshot just before attempting to add a GV account.
dziny
join:2015-06-22

2 recommendations

dziny

Member

Yes it's definitely fast enough for an asterisk-based PBX. I use the free offer to run my own us-based vpn server.

RobThompson
Caution - VoIP Challenged Alert
Premium Member
join:2012-02-14
J8G 0C9

1 edit

1 recommendation

RobThompson to Stewart

Premium Member

to Stewart
Hi Stewart:

I am getting an error after: "... Give it a name e.g. sip. Under Target tags, type a tag name, e.g. sip.
Under Specified protocols and ports enter "udp:5060; udp:5160; udp:10000-20000" (without the quotes and assuming default FreePBX settings). Click Save."

Any ideas?

Thanks,

PS: I used 35.2nn.nnn.0/25

Rob.
Stewart
join:2005-07-13

1 edit

Stewart

Member

Sorry, I missed that. Under Source IP ranges, enter 0.0.0.0/0 or whatever restricted range you desire.

AllThumbs
join:2006-02-07
Charleston, SC

AllThumbs to Stewart

Member

to Stewart
Great work, Stewart See Profile. Can't wait to try it.
phonesimon
join:2014-10-08
Pennsylvania

2 recommendations

phonesimon to Stewart

Member

to Stewart
Google's cloud servers are super fast. But if you saturate the CPU for a short period of time you get throttled way down and audio is ruined. So you have to be very careful not to let the CPU get to 100% for any noticeable amount of time.

AllThumbs
join:2006-02-07
Charleston, SC

AllThumbs

Member

We've experienced the same performance issues with other Google Cloud apps. That obviously will ruin a PBX so I'm sorry to hear about the throttling. We're working on what we believe will be a great cloud alternative with lots of RAM and great performance. Still shooting for a turnkey PBX setup and $1 a month for the cloud hosting. We're not quite there yet, but we're close. Can't wait to compare with Google Cloud alternative. More details soon.
carlm
join:2014-09-29
united state

1 edit

2 recommendations

carlm to phonesimon

Member

to phonesimon
said by phonesimon:

Google's cloud servers are super fast. But if you saturate the CPU for a short period of time you get throttled way down and audio is ruined. So you have to be very careful not to let the CPU get to 100% for any noticeable amount of time.

From Google's description that's not the way the f1-micro and Google's other shared core machines work:

»cloud.google.com/compute ··· hinetype :

"Note: f1-micro instances get 0.2 of a vCPU and are allowed to burst up to a full vCPU for short periods. g1-small instances get 0.5 of a vCPU and are allowed to burst up to a full vCPU for short periods."

For the f1-micro, if you know your PBX will never need more than 0.2 vCPU you should be OK. A personal PBX that never handles more than a few concurrent calls might fall into this category. If you are a service provider, on the other hand, you could test with a few concurrent calls and then put it into production and have it fail miserably.

It should be possible to measure CPU utilization by running 'top' or similar from the command line. With your worst case workload the CPU idle time should always be in excess of, say, 85%, allowing a cushion for background tasks.

If you ever have a workload that requires any CPU bursts, you can't count on its working all the time.

[ EDIT ]

The comments above apply only to real time workloads such as VoIP.
Stewart
join:2005-07-13

5 recommendations

Stewart to phonesimon

Member

to phonesimon
said by phonesimon:

Google's cloud servers are super fast. But if you saturate the CPU for a short period of time you get throttled way down and audio is ruined. So you have to be very careful not to let the CPU get to 100% for any noticeable amount of time.

I wrote a simple script to evaluate throttling:
#!/usr/bin/perl
 
$start = time;
while (1) {
	$oldt = time;
	$cnt = 0;
	while ($oldt == time) {
		for ($i = 0; $i < 1000000; ++$i) {}
		++$cnt;
	}
	print time - $start, " $cnt\n";
}
 
My conclusion is that they use a token bucket algorithm. The bucket has a capacity of 24 core-seconds and is refilled at 0.2 core-seconds per second, i.e. after being idle for at least two minutes, you can use a full core (if available) for 30 seconds. The practical impact on a small PBX is that you can update extensions, routes, etc., without impacting the audio, but if you try to build something you are in trouble.

However, GCP is extremely flexible and there are many ways to handle this. First, you can simply increase the machine capacity when needed. This unfortunately requires a restart, obviously dropping any calls in progress. So, on a day when you want to work on the PBX, you change to a two-core machine before the business opens and put it back to micro after it closes. This costs less than $1. Better, take a snapshot before the business opens and restart the production PBX. You can then create a new two-core instance from the snapshot and work in that, without risk of disrupting normal traffic. If the updates are deemed successful, downgrade the machine and reassign the static IP to the new instance. Once working, you can delete the old one. The whole sequence also costs less than a buck.
carlm
join:2014-09-29
united state

carlm

Member

said by Stewart:

...after being idle for at least two minutes, you can use a full core (if available) for 30 seconds.

I'm sure Stewart gets this, but I just want to expand on "if available".
A fully subscribed f1-micro vCPU has 5 tenants. If the other 4 tenants are using 20% CPU each, then you can't get any CPU burst no matter how long you have been idle. That's because there are no idle cycles to give you.

P.S. Thanks for the info, Stewart.
Stewart
join:2005-07-13

1 recommendation

Stewart

Member

said by carlm:

A fully subscribed f1-micro vCPU has 5 tenants.

I don't believe that's the case. The machine likely has four physical CPUs, each with four hyperthreaded cores, i.e. 32 logical CPUs. Assuming that this is loaded with only 160 tenants, the statistics are overwhelmingly in favor of being able to get some compute when needed.
carlm
join:2014-09-29
united state

1 recommendation

carlm

Member

said by Stewart:

said by carlm:

A fully subscribed f1-micro vCPU has 5 tenants.

I don't believe that's the case. The machine likely has four physical CPUs, each with four hyperthreaded cores, i.e. 32 logical CPUs. Assuming that this is loaded with only 160 tenants, the statistics are overwhelmingly in favor of being able to get some compute when needed.

I think you're right, basically. That value of 160 needs to be reduced for the hypervisor (1 physical core?) and increased in the case of hex core physical CPUs.

The conclusions are:

1. It's more likely to be able to get some CPU burst.
2. It's less likely to be able to burst to 100%.
3. Any real time workload that requires any CPU burst cannot be counted on to work all the time (still true for your model).

Now I'm wondering how one would determine how much physical core share his VM is using. What would 'top' report for idle % when his VM was using its full 20% share? 0%? 80%?
Stewart
join:2005-07-13

Stewart

Member

said by carlm:

What would 'top' report for idle % when his VM was using its full 20% share? 0%? 80%?

A quick test showed that idle displays at 0 when the script is running, whether throttled or not. Before throttling, user time is near 100% and system time near 0. After throttling, user time is ~98.4% and system is ~1.6%.

So, it appears that top's display is: idle, % of real time not attempting to run; user, % of real time attempting to run and not in a system call; system, % of real time with system call pending.
Stewart

1 recommendation

Stewart to carlm

Member

to carlm
said by carlm:

1. It's more likely to be able to get some CPU burst.
2. It's less likely to be able to burst to 100%.
3. Any real time workload that requires any CPU burst cannot be counted on to work all the time (still true for your model).

OK, so I set up another simple script:
#!/usr/bin/perl
 
$| = 1;
sleep 1 unless time % 6;
$mincnt = 99;
 
while (1) {
    while (time % 6) { select(undef, undef, undef, 0.05); }
    $cnt = 0;
    while (time % 6 == 0) {
        for ($i = 0; $i < 1000000; ++$i) {}
        ++$cnt;
    }
    $mincnt = $cnt if $cnt < $mincnt;
    print scalar(gmtime), " $cnt $mincnt\n";
}
 
This attempts to get CPU for the first second of each six, reporting a count and tracking the worst case (minimum). It's been running now for 8+ hours on micro instances in five regions (the three free-eligible ones, London and Singapore). Across the 24,000+ attempts, the lowest measurement was 0.5 core-seconds (vs 0.2 core-seconds 'guaranteed'). Compared to low-end VPSes, this is quite impressive.

Besides, I'm sure that the smart folks at 3CX analyzed this very carefully before offering their free trials in the Google Cloud. If performance wasn't up to snuff, not only would trial users not upgrade to the paid product, but it would damage the reputation of the product in general.
twinclouds
join:2010-06-12
San Diego, CA

twinclouds to Stewart

Member

to Stewart
Hi, Stewart:
I signed a GCP account and created a VM instances. Looks like I can only use the browser based terminal but not Putty, is that right? I installed Asterisk 15 with out GUI on it with no problem. However, I could not access the SIP port from my desktop even though I created the Firewall rules with "udp:5060; udp:10000-20000". Do know what is the reason and how I can make it work?
Because I don't use free PBX, some of your instructions does not apply to me.
One more thing, if I change the ip address from Ephemeral to Static, looks like it will incur constant charges in the "almost free" case?
Thank you in advance for your help.
Stewart
join:2005-07-13

1 edit

1 recommendation

Stewart

Member

Some ways to use PuTTY: Create key pair with PuTTYgen. Put public key in this format:
ssh-rsa (long string of base64 starting with AAA and ending with ==) username
 
Edit the instance and under SSH keys, paste the line above. Click Save.
This will create /home/username with the appropriate data.
In PuTTY, configure to use the corresponding private key and the same username.

Or, using the browser based terminal, add a user and set up the .ssh and authorized_keys files manually.

Or, edit /etc/ssh/sshd_config to allow logging in with a password and/or to permit root logins, then restart the sshd service.

Static IP addresses are charged for only when not connected to a running instance. Assuming that your PBX will run 24/7, that shouldn't be an issue.

I have no idea what could be wrong with the firewall. You could run tcpdump on the instance to confirm that it's a firewall issue (rather than Asterisk not listening or not processing the request correctly).

Things to check on firewall:

1. In the Instance details, check that your sip (or whatever you called it) tag appears under Network tags.
2. Under VPC network -> Firewall rules -> your rule , confirm that the same tag name appears in Target tags, Source filters -> IP ranges has 0.0.0.0/0 (or something that matches your source) and that Protocols and ports shows udp:5060 and udp:10000-20000.

Edit: Note that the instance runs behind a NAT and you must configure Asterisk accordingly.
carlm
join:2014-09-29
united state

1 edit

1 recommendation

carlm to Stewart

Member

to Stewart
said by Stewart:

Across the 24,000+ attempts, the lowest measurement was 0.5 core-seconds (vs 0.2 core-seconds 'guaranteed'). Compared to low-end VPSes, this is quite impressive.

Yes, that's good performance, but:

Like any good engineer I do worst case design. I would need to determine that with my heaviest workload there are no media problems when I'm getting the guaranteed 0.2 core-seconds and no more than that.

I've spent some time spelunking through the GCP Compute Engine doc. I could have missed it, but I didn't find any way to disable bursting (for test purposes). I also didn't find any way to measure peak CPU utilization. The only CPU utilization available (in the Compute Engine Dashboard) seems to be 1-second 1-minute average, which doesn't help.

I guess it was foolish of me to think of using 'top' within the VM -- the VM is oblivious to what goes on outside of it. Only the hypervisor would know how much CPU share a VM is using (unless I'm missing something).

P.S. I have to give Google credit for specifying a guaranteed minimum CPU share. With Digital Ocean and RamNode, who knows?
carlm

2 edits

carlm to Stewart

Member

to Stewart
said by Stewart:

Note that the instance runs behind a NAT and you must configure Asterisk accordingly.

Are you sure? I didn't see any mention of NAT here:

»cloud.google.com/compute ··· dresses/

Scratch that. Dumb question. A VPS host has lots and lots of virtual network interfaces.
dziny
join:2015-06-22

1 recommendation

dziny

Member

You definitely are. If you do ifconfig you'll see your interface has ip address 10.xx.xx.xx so in a private range.
carlm
join:2014-09-29
united state

2 edits

carlm

Member

And you're not using load balancing?

Scratch that. Dumb question. A VPS host has lots and lots of virtual network interfaces.
twinclouds
join:2010-06-12
San Diego, CA

twinclouds to Stewart

Member

to Stewart
Thanks. It works now. The reason was that I opened the ports on egress but actually I need to open the ports on ingress.
Right now, I have one way audio problem. Maybe because of NAT? I have already set the nat=yes in sip.conf
Stewart
join:2005-07-13

1 recommendation

Stewart to carlm

Member

to carlm
said by carlm:

Like any good engineer I do worst case design.

Unfortunately, any VoIP over the public internet is not a worst case design. After accumulating 24 hours run time, I measured 0.46 core-seconds minimum on Oregon and Singapore; the other three remained at 0.5.

For most small business users, if there is a little voice breakup on the rare occasions when you make changes to the PBX config, it's not a big deal.

However, you should determine that 0.2 is enough for your worst case continuous load. If not transcoding, IMO that's about 9 concurrent calls. With transcoding to/from Opus or another high complexity codec, it could be as few as 2.
phonesimon
join:2014-10-08
Pennsylvania

5 recommendations

phonesimon

Member

It seems that in so many cases the cost of "free" in time and fretting exceeds the $5/month of a Digital Ocean VPS or similar. In this thread alone there's probably a couple hundred dollars worth of engineering labor trying to figure out the specs and boundaries of the Google cloud service.
Stewart
join:2005-07-13

Stewart to twinclouds

Member

to twinclouds
said by twinclouds:

Right now, I have one way audio problem.

Do you have the correct public IP address values in external_media_address, external_signaling_address and/or externip in the appropriate pjsip / sip config files?

Is that's not the issue, run tcpdump on the instance and see where RTP is coming from and where it's going.
carlm
join:2014-09-29
united state

carlm to Stewart

Member

to Stewart
said by Stewart:

However, you should determine that 0.2 is enough for your worst case continuous load.

I agree, but how do I determine that? I'd like to do an actual test but I can't control how many CPU-seconds I get. I'd rather not assume that I'm getting ~0.5 and extrapolate. Too bad I can't turn off bursting.
carlm

1 recommendation

carlm to phonesimon

Member

to phonesimon
said by phonesimon:

It seems that in so many cases the cost of "free" in time and fretting exceeds the $5/month of a Digital Ocean VPS or similar.

The $5/month Digital Ocean VPS is also shared core with bursting. We just don't know whether there's a minimum CPU-seconds, and if so, what it might be. Thus the same concern applies to Digital Ocean, as far as I can see.
twinclouds
join:2010-06-12
San Diego, CA

twinclouds to Stewart

Member

to Stewart
Hi, Stewart:
I am using sip.conf so I tried both externip and externhost (I assigned the external address a host name using my ddns server.) but neither worked. Then I found I also need to specify the internal ip address as the localnet ip address. After doing that using either externip and externhost will work.
Looks like the Asterisk PBX is working now. Thank you very much for your help.
phonesimon
join:2014-10-08
Pennsylvania

1 recommendation

phonesimon to carlm

Member

to carlm
DO promises a minimum of 1 "vCPU" and at least from the operating system's perspective I can run at sustained near-100% indefinitely without any problems. So I don't see how these are the same.
carlm
join:2014-09-29
united state

2 recommendations

carlm

Member

said by phonesimon:

DO promises a minimum of 1 "vCPU"...

Not with a Standard Droplet. Only with an Optimized Droplet (about twice as expensive).

»www.digitalocean.com/com ··· lication :

"Additionally, these [ Optimized ] Droplets are engineered to have dedicated underlying physical resources so that other guests on the same hardware very rarely impact your Droplet performance. Each vCPU on an Optimized Droplet maps directly to a dedicated hyper-thread on the underlying physical CPU."

A Standard Droplet doesn't have a dedicated hyperthread: it's shared core, just like GCP shared core f1-micro and g1-small.

When you say "run at sustained near-100%", if you're looking at CPU utilization inside the VM, rather than in some DO control panel, 100% represents 100% of whatever CPU-seconds DO is giving your VM at that point in time, not 100% of wall clock time.

It would be interesting to know the minimum CPU-seconds for DO shared core Standard Droplets. GCP seems to be the only platform that provides this info.

AllThumbs
join:2006-02-07
Charleston, SC

1 recommendation

AllThumbs to Stewart

Member

to Stewart
said by Stewart:

I'm sure that the smart folks at 3CX analyzed this very carefully before offering their free trials in the Google Cloud. If performance wasn't up to snuff, not only would trial users not upgrade to the paid product, but it would damage the reputation of the product in general.

Based purely on subjective observation, I would submit that the 3CX platform is considerably more efficient than a traditional Asterisk/FreePBX setup because it was engineered to perform specific tasks rather than providing "kitchen sink" functionality for anything a developer could potentially dream up. Bottom line: I don't think you can or should judge the probable success of Asterisk/FreePBX servers on the platform based upon the fact that the platform works well for 3CX. It's apples and oranges IMHO.