postscreen request: pcre support

[Jeroen Koekkoek]

Hi,

I would like to request pcre table support in postscreen for some fields
e.g. client_name, helo_name, etc.

For example if client is not listed on any dnsbl, but the reverse
hostname matches /\.dsl\./, the client is greylisted.

Or if client is listed on a single dnsbl and contains something like
dialup, the connection is dropped.


The option could be called something like postscreen_client_name_maps
and let users add a pcre table the way they are used to.

postscreen_client_name_maps = pcre:/etc/postfix/dslhosts

The file could contain something like:

/\.dsl\./ <score>

Additional maps could be added like, postscreen_helo_name_maps, etc.
Another postscreen check could be postscreen_helo_name_empty followed by
a score.


I haven't looked at the postscreen code yet, so I wouldn't know how hard
this is to implement. But I think it would be really usefull.

Best regards,
Jeroen Koekkoek

[Wietse Venema]

Post by Jeroen Koekkoek
Hi,
I would like to request pcre table support in postscreen for some fields
e.g. client_name, helo_name, etc.
For example if client is not listed on any dnsbl, but the reverse
hostname matches /\.dsl\./, the client is greylisted.
Or if client is listed on a single dnsbl and contains something like
dialup, the connection is dropped.

This functionality already exists in smtpd. There is no need to duplicate
this in postscreen.

Postscreen's purpose is to keep zombies away so that you can keep
using the existing smtpd features.

It is not a scoring system that makes a decision at the end.
Instead, postscreen makes the decision as early as possible.

Wietse

[Mark Scholten]

-----Original Message-----
Sent: Wednesday, December 01, 2010 4:41 PM
To: Postfix users
Subject: Re: postscreen request: pcre support

Post by Jeroen Koekkoek
Hi,
I would like to request pcre table support in postscreen for some

fields

Post by Jeroen Koekkoek
e.g. client_name, helo_name, etc.
For example if client is not listed on any dnsbl, but the reverse
hostname matches /\.dsl\./, the client is greylisted.
Or if client is listed on a single dnsbl and contains something like
dialup, the connection is dropped.

This functionality already exists in smtpd. There is no need to
duplicate
this in postscreen.

Is it possible to use postscreen and give points for things and when it is
between 2 values it is greylisted and if it is below the lowest value it is
accepted and if it is above the highest value it is rejected? In that case
it would be useful to have an option to have pcre table to give certain
points. If you are listed in dnsbl X and you've a hostname that match
/\.dsl\./ -> greylist, if you are only listed in dnsbl X -> accept, if you
are listed in dnsbl X, you've a hostname that match /\.dsl\./ and are listed
in dnsbl Y -> reject.

Would something like that be possible with postfix/postscreen without using
a milter?

Mark

[j***@intuxicated.org]

On Wed, 1 Dec 2010 10:41:22 -0500 (EST), Wietse Venema

Post by Wietse Venema

Post by Jeroen Koekkoek
Hi,
I would like to request pcre table support in postscreen for some fields
e.g. client_name, helo_name, etc.
For example if client is not listed on any dnsbl, but the reverse
hostname matches /\.dsl\./, the client is greylisted.
Or if client is listed on a single dnsbl and contains something like
dialup, the connection is dropped.

This functionality already exists in smtpd. There is no need to duplicate
this in postscreen.
Postscreen's purpose is to keep zombies away so that you can keep
using the existing smtpd features.
It is not a scoring system that makes a decision at the end.
Instead, postscreen makes the decision as early as possible.
Wietse

Not entirely, because I can't combine scores in smtpd. I would need a
policy service for that (correct me if i'm wrong). So if I wanted to do
this check I would need an smtpd + policy service and the policy service
would need to do the exact same lookups in order to get a combined score
and make a descision based on that.

I think it's a lot of overhead where one or two pcre checks would suffice.

If I create a patch, could this feature make its way into postfix?

Jeroen

[Wietse Venema]

Post by j***@intuxicated.org
On Wed, 1 Dec 2010 10:41:22 -0500 (EST), Wietse Venema

Post by Wietse Venema

Post by Jeroen Koekkoek
Hi,
I would like to request pcre table support in postscreen for some fields
e.g. client_name, helo_name, etc.
For example if client is not listed on any dnsbl, but the reverse
hostname matches /\.dsl\./, the client is greylisted.
Or if client is listed on a single dnsbl and contains something like
dialup, the connection is dropped.

This functionality already exists in smtpd. There is no need to duplicate
this in postscreen.
Postscreen's purpose is to keep zombies away so that you can keep
using the existing smtpd features.
It is not a scoring system that makes a decision at the end.
Instead, postscreen makes the decision as early as possible.
Wietse

Not entirely, because I can't combine scores in smtpd. I would need a
policy service for that (correct me if i'm wrong). So if I wanted to do
this check I would need an smtpd + policy service and the policy service
would need to do the exact same lookups in order to get a combined score
and make a descision based on that.

Again. if something can already be done with smtpd plus milter or
policy plugin or content filter then I urge you to keep using that
already existing functionality.

Post by j***@intuxicated.org
I think it's a lot of overhead where one or two pcre checks would

suffice. > > If I create a patch, could this feature make its way
into postfix? > > Jeroen > >

I don't take any code before I have seen a clear design of user
interface (how to use) and semantics (what it does). That is,
write the manpage and we can talk about how it would work. But I
warn you, I will not take something that simply hard-codes PCRE
lookups plus counter into postscreen.

Wietse

[Len Conrad]

Post by j***@intuxicated.org
Not entirely, because I can't combine scores in smtpd.

postfwd policy service can weight and score.

Len

[Tomoyuki Murakami]

Post by Wietse Venema
Again. if something can already be done with smtpd plus milter or
policy plugin or content filter then I urge you to keep using that
already existing functionality.

and also said,

Post by Wietse Venema
Postscreen's purpose is to keep zombies away so that you can keep
using the existing smtpd features.
It is not a scoring system that makes a decision at the end.
Instead, postscreen makes the decision as early as possible.

To keeping away zombies from smtpd, and do the decision as early
as possible, it is natural that the similar access control
functionality as implemented in smtpd may be required, I think.
because zombies can be fixed their behavior over time,
and the difference between legitimates may be just their IPs
accesing from and reverse DNS names. so that the RBLDNS
scoreing works very effectively in my site now.

If there are any additional features to the postscreen,
at least, policy-delegation I/F is useful for that purpose.

--
Tomo.

[Jeroen Koekkoek]

Post by Wietse Venema

Post by j***@intuxicated.org
On Wed, 1 Dec 2010 10:41:22 -0500 (EST), Wietse Venema

Post by Wietse Venema

Post by Jeroen Koekkoek
Hi,
I would like to request pcre table support in postscreen for some fields
e.g. client_name, helo_name, etc.
For example if client is not listed on any dnsbl, but the reverse
hostname matches /\.dsl\./, the client is greylisted.
Or if client is listed on a single dnsbl and contains something like
dialup, the connection is dropped.

This functionality already exists in smtpd. There is no need to duplicate
this in postscreen.
Postscreen's purpose is to keep zombies away so that you can keep
using the existing smtpd features.
It is not a scoring system that makes a decision at the end.
Instead, postscreen makes the decision as early as possible.
Wietse

Not entirely, because I can't combine scores in smtpd. I would need a
policy service for that (correct me if i'm wrong). So if I wanted to do
this check I would need an smtpd + policy service and the policy service
would need to do the exact same lookups in order to get a combined score
and make a descision based on that.

Again. if something can already be done with smtpd plus milter or
policy plugin or content filter then I urge you to keep using that
already existing functionality.

Post by j***@intuxicated.org
I think it's a lot of overhead where one or two pcre checks would

suffice.> > If I create a patch, could this feature make its way
into postfix?> > Jeroen> >
I don't take any code before I have seen a clear design of user
interface (how to use) and semantics (what it does). That is,
write the manpage and we can talk about how it would work. But I
warn you, I will not take something that simply hard-codes PCRE
lookups plus counter into postscreen.
Wietse

I've read through the postscreen code and got a general understanding of
how it works internally. But judging from the documentation: is
postscreen intended to ever do more than allowing/disallowing client
connections? e.g. greylisting or specifying a follow-up service like
postgrey?

If it's not: It would be nice if the dnsbl results could be passed to
the follow-up smtpd process, so they in turn can be passed to a policy
daemon. It would save cpu cycles, etc and it would make implementing a
policy daemon that needs those results anyway a lot easier.

If it is: I'll write about how I think the configuration options, maps,
etc should look.

- Jeroen

[Victor Duchovni]

Post by Jeroen Koekkoek
I've read through the postscreen code and got a general understanding of
how it works internally. But judging from the documentation: is postscreen
intended to ever do more than allowing/disallowing client connections? e.g.
greylisting or specifying a follow-up service like postgrey?
If it's not: It would be nice if the dnsbl results could be passed to the
follow-up smtpd process, so they in turn can be passed to a policy daemon.
It would save cpu cycles, etc and it would make implementing a policy
daemon that needs those results anyway a lot easier.

No, for most connections that are passed on the client was already
whitelisted, so no tests were performed and there are no results to
pass on. There is no benefit in over-optimizing this code path, if
DNSbl lookups were performed, the data is cached in your DNS cache,
and another lookup will be quite cheap.

As for policy daemons, they may run before or after Postfix performs
any RBL lookups, and the complexity of trying to pass such data to
them is not worth the effort.

[Jeroen Koekkoek]

Post by Victor Duchovni

Post by Jeroen Koekkoek
I've read through the postscreen code and got a general understanding of
how it works internally. But judging from the documentation: is postscreen
intended to ever do more than allowing/disallowing client connections? e.g.
greylisting or specifying a follow-up service like postgrey?
If it's not: It would be nice if the dnsbl results could be passed to the
follow-up smtpd process, so they in turn can be passed to a policy daemon.
It would save cpu cycles, etc and it would make implementing a policy
daemon that needs those results anyway a lot easier.

No, for most connections that are passed on the client was already
whitelisted, so no tests were performed and there are no results to
pass on. There is no benefit in over-optimizing this code path, if
DNSbl lookups were performed, the data is cached in your DNS cache,
and another lookup will be quite cheap.
As for policy daemons, they may run before or after Postfix performs
any RBL lookups, and the complexity of trying to pass such data to
them is not worth the effort.

You are right, forgot about postscreen's internal cache.

Well, not before the connection is handed of to an smtpd right? But yes,
you are right. Results should be read from a caching dns server to keep
things simple.

- Jeroen