Rate-limiting OUTGOING smtp?

[Gary Oberbrunner]

I'm running postfix 2.2-20041212, and my ISP will get mad if I send too
many emails per hour. I'd like to rate-limit my outgoing smtp
connections so I don't send more than N messages per hour. I see there
is an 'anvil' process which can be used for this, but AFAICT only smtpd
uses it (to limit incoming connections), not smtp. I don't care how
many incoming mails I get (they come from my local users only on my
trusted internal net); I just want them to queue up and drain out at the
proper rate.

thanks,

-- Gary Oberbrunner

[Victor Duchovni]

Post by Gary Oberbrunner
I'm running postfix 2.2-20041212, and my ISP will get mad if I send too
many emails per hour. I'd like to rate-limit my outgoing smtp
connections so I don't send more than N messages per hour. I see there
is an 'anvil' process which can be used for this, but AFAICT only smtpd
uses it (to limit incoming connections), not smtp. I don't care how
many incoming mails I get (they come from my local users only on my
trusted internal net); I just want them to queue up and drain out at the
proper rate.

Get a new ISP or a better class of service. You could try putting all mail
on HOLD and scheduling it manually, but this really stinks. You could
also try spawn at most one smtp(8) client every 60/N minutes, and give
it a max_use of 1. This requires a delay of 3600/N seconds during smtp(8)
process startup.

# At most 100 smtp(8) deliveries per hour
debugger_command = /bin/sleep 36

and a "-D" flag appended to th "smtp unix ... smtp" master.cf entry,
could do the trick (still ugly).

smtp unix ... smtp -D
-o max_use=1

the above spaces out all deliveries for the transport in question at
least 36 seconds apart.

[Perry Smith]

Gary,

Is it possible that you misunderstand the rules of the ISP? The usual
customer will send to the ISP's smtp daemons which causes a load to
them. You are not, You should be connecting directly to the
destination smtp daemons and the local ISP would have a fairly rough
time even detecting it and counting it. To them, it is just another IP
connection going through their site (it just happens to be port 25 but
why would they care?)

The exception to this would be if you are passing mail to the ISP's
smtp daemons to act as a relay for firewall concerns or similar things.

Post by Victor Duchovni

Post by Gary Oberbrunner
I'm running postfix 2.2-20041212, and my ISP will get mad if I send
too
many emails per hour. I'd like to rate-limit my outgoing smtp
connections so I don't send more than N messages per hour. I see
there
is an 'anvil' process which can be used for this, but AFAICT only
smtpd
uses it (to limit incoming connections), not smtp. I don't care how
many incoming mails I get (they come from my local users only on my
trusted internal net); I just want them to queue up and drain out at
the
proper rate.

Get a new ISP or a better class of service. You could try putting all
mail
on HOLD and scheduling it manually, but this really stinks. You could
also try spawn at most one smtp(8) client every 60/N minutes, and give
it a max_use of 1. This requires a delay of 3600/N seconds during
smtp(8)
process startup.
# At most 100 smtp(8) deliveries per hour
debugger_command = /bin/sleep 36
and a "-D" flag appended to th "smtp unix ... smtp" master.cf entry,
could do the trick (still ugly).
smtp unix ... smtp -D
-o max_use=1
the above spaces out all deliveries for the transport in question at
least 36 seconds apart.
--
Viktor.
Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.
To unsubscribe from the postfix-users list, visit

[Cami]

Post by Gary Oberbrunner
I'm running postfix 2.2-20041212, and my ISP will get mad if I send too
many emails per hour. I'd like to rate-limit my outgoing smtp
connections so I don't send more than N messages per hour.

Connections really dont have anything to do with rate limiting messages.

I've written a Policy Daemon which does do rate limiting of messages
per hour (or mail volume) on a per user basis.

This should sort out your problem.

Cami

[Cami]

Post by Cami

Post by Gary Oberbrunner
I'm running postfix 2.2-20041212, and my ISP will get mad if I send
too many emails per hour. I'd like to rate-limit my outgoing smtp
connections so I don't send more than N messages per hour.

Connections really dont have anything to do with rate limiting messages.
I've written a Policy Daemon which does do rate limiting of messages
per hour (or mail volume) on a per user basis.
This should sort out your problem.

Actually, this wont sort out your problem. I'll be
changing the code shortly to do a HOLD instead of
DEFER. But this really needs a newer module so that
it is applied to *all* mail and not just on a per
sender basis.

Only problem is that this will also affect `incoming`
mail as that will also get added to the 'max messages'
count. One could have a configuration option that
specifies what ip ranges would be used for this
option.

Comments/Ideas?

Cami

[Victor Duchovni]

Post by Cami
Only problem is that this will also affect `incoming`
mail as that will also get added to the 'max messages'
count. One could have a configuration option that
specifies what ip ranges would be used for this
option.
Comments/Ideas?

If the OP wants to rate limit *output* to a particular nexthop (even if
it is relayhost), rate limiting input may solve the wrong problem.

[Cami]

Post by Victor Duchovni

Post by Cami
Only problem is that this will also affect `incoming`
mail as that will also get added to the 'max messages'
count. One could have a configuration option that
specifies what ip ranges would be used for this
option.

If the OP wants to rate limit *output* to a particular nexthop (even if
it is relayhost), rate limiting input may solve the wrong problem.

Good point. I wont be doing anything untill the
OP has made clear exactly what he is after.

Cami

[Gary Oberbrunner]

Post by Cami

Post by Victor Duchovni
If the OP wants to rate limit *output* to a particular nexthop (even if
it is relayhost), rate limiting input may solve the wrong problem.

Good point. I wont be doing anything untill the
OP has made clear exactly what he is after.

OK, here's the whole story. ISP is Verizon Business DSL, dynamic IP.
Verizon's AUP at http://www2.verizon.net/policies/email.asp says no more
than 500 outbound emails per hour, whether sent through their server or
any other way (i.e. direct SMTP) -- at least that's what they say (I
called them to confirm). If we violate that we could be in trouble.
Yes, it's draconian, we all have to suffer because of a few evil
spammers. :-(

Now of course they wouldn't know if we sent emails directly, as Perry
Smith noted. It'd just look like regular traffic. However, I want to
be good if I can, and what's more serious, our dynamic IP addr is
blocked by most RBLs, so sending directly is really not very practical
(I tried using a transport map for the blocking domains and relaying
only those through the ISP, but maintaining that is a real pain. It's
just too big.) It's simplest to relay everything through the ISP's
server. (I could get a different email-only ISP; I'd like to avoid that
if possible. Another option is a static IP which might not be blocked,
so I could send directly.)

Anyway, that's why I want to rate-limit my outgoing emails to no more
than 500 per hour and queue up the rest. Can I do it?

Thanks,

-- Gary

[Victor Duchovni]

Post by Gary Oberbrunner
Anyway, that's why I want to rate-limit my outgoing emails to no more
than 500 per hour and queue up the rest. Can I do it?

Did you read the answer with the "debugger_command = sleep 36" (for 100
msgs/hour, use "sleep 8" for 450/hour) and "-o max_use=1" for smtp(8).

[Gary Oberbrunner]

Post by Victor Duchovni

Post by Gary Oberbrunner
Anyway, that's why I want to rate-limit my outgoing emails to no more
than 500 per hour and queue up the rest. Can I do it?

Did you read the answer with the "debugger_command = sleep 36" (for 100
msgs/hour, use "sleep 8" for 450/hour) and "-o max_use=1" for smtp(8).

Yes, I think I'll try that method. In my ideal world, all msgs would go
out immediately until the limit was close and only *then* would it
throttle, but hey, I'll take what I can get. :-)

thanks, Victor!

-- Gary

[Victor Duchovni]

Post by Gary Oberbrunner

Post by Victor Duchovni

Post by Gary Oberbrunner
Anyway, that's why I want to rate-limit my outgoing emails to no more
than 500 per hour and queue up the rest. Can I do it?

Did you read the answer with the "debugger_command = sleep 36" (for 100
msgs/hour, use "sleep 8" for 450/hour) and "-o max_use=1" for smtp(8).

Yes, I think I'll try that method. In my ideal world, all msgs would go
out immediately until the limit was close and only *then* would it
throttle, but hey, I'll take what I can get. :-)

This gives you a minimum spacing between messages of 8s, if you generate
a burst of 100 messages, it will take 800 seconds (+ delivery latency)
to push them out, since all mail will go to a single network
proximate relayhost, the latency should be consistently low. A 15 minute
delay for a burst of 100 messages is not too bad given your limit of 500
per hour.

Instead of using "sleep" and debugger_command, you can also set the
relayhost to a non_responsive IP address, set the process limit in
master.cf to 1 and the smtp_connect_timeout to 8s. Finally configure
fallback_relay = [real-relayhost]. This avoids the fork/exec of the
sleep command, but is arguably more obscure. There is at this time no
direct mechanism for delayed transmission.

[Gary Oberbrunner]

Post by Victor Duchovni
This gives you a minimum spacing between messages of 8s, if you generate
a burst of 100 messages, it will take 800 seconds (+ delivery latency)
to push them out, since all mail will go to a single network
proximate relayhost, the latency should be consistently low. A 15 minute
delay for a burst of 100 messages is not too bad given your limit of 500
per hour.

Yeah, I think it's acceptable.

Post by Victor Duchovni
Instead of using "sleep" and debugger_command, you can also set the
relayhost to a non_responsive IP address, set the process limit in
master.cf to 1 and the smtp_connect_timeout to 8s. Finally configure
fallback_relay = [real-relayhost]. This avoids the fork/exec of the
sleep command, but is arguably more obscure.

^^^^^^^^^^^^^^^^^^^^^
Whoa, I'll say so! I think I'll stick with your original solution.

-- Gary