PATCH: postscreen contantly deferring mail

Discussion:

(too old to reply)

Wietse Venema

2016-07-28 23:19:02 UTC

I have uploaded a patch that makes postscreen share test results
between concurrent connections from the same IP address. This patch
requires Postfix 3.1 or later.

ftp://ftp.porcupine.org/mirrors/postfix-release/experimental/feature-patches/
-r--r--r-- 1 wietse wietse 14473 Jul 28 18:48 20160728-postscreen-3.1-3.2.patch
-r--r--r-- 1 wietse wietse 480 Jul 28 19:06 20160728-postscreen-3.1-3.2.patch.gpg1
-r--r--r-- 1 wietse wietse 220 Jul 28 19:06 20160728-postscreen-3.1-3.2.patch.gpg2
-r--r--r-- 1 wietse wietse 280 Jul 28 19:06 20160728-postscreen-3.1-3.2.patch.sig

Wietse

20160728

Bugfix (introduced: 20090614): with concurrent connections
from the same client IP address, and after-220 tests enabled,
postscreen could overwrite the cached "all tests completed"
result for one connection that completed the after-220 tests,
with the "some tests not completed" result for a concurrent
connection where the client hung up before completing the
after-220 tests. Files: postscreen_misc.c, postscreen_state.c,
postscreen.h, postscreen_tests.c, postscreen.c, postscreen_smtpd.c,
postscreen_early.c.

Wietse Venema

2016-07-28 23:21:27 UTC

Permalink

Usage:

$ cd $POSTFIX-SOURCE
$ patch -p1 <20160728-postscreen-3.1-3.2.patch

Wietse

Post by Wietse Venema
20160728
Bugfix (introduced: 20090614): with concurrent connections
from the same client IP address, and after-220 tests enabled,
postscreen could overwrite the cached "all tests completed"
result for one connection that completed the after-220 tests,
with the "some tests not completed" result for a concurrent
connection where the client hung up before completing the
after-220 tests. Files: postscreen_misc.c, postscreen_state.c,
postscreen.h, postscreen_tests.c, postscreen.c, postscreen_smtpd.c,
postscreen_early.c.

Noel Jones

2016-07-29 17:09:32 UTC

Permalink

Post by Wietse Venema
I have uploaded a patch that makes postscreen share test results
between concurrent connections from the same IP address. This patch
requires Postfix 3.1 or later.
ftp://ftp.porcupine.org/mirrors/postfix-release/experimental/feature-patches/
-r--r--r-- 1 wietse wietse 14473 Jul 28 18:48 20160728-postscreen-3.1-3.2.patch
-r--r--r-- 1 wietse wietse 480 Jul 28 19:06 20160728-postscreen-3.1-3.2.patch.gpg1
-r--r--r-- 1 wietse wietse 220 Jul 28 19:06 20160728-postscreen-3.1-3.2.patch.gpg2
-r--r--r-- 1 wietse wietse 280 Jul 28 19:06 20160728-postscreen-3.1-3.2.patch.sig
Wietse
20160728
Bugfix (introduced: 20090614): with concurrent connections
from the same client IP address, and after-220 tests enabled,
postscreen could overwrite the cached "all tests completed"
result for one connection that completed the after-220 tests,
with the "some tests not completed" result for a concurrent
connection where the client hung up before completing the
after-220 tests. Files: postscreen_misc.c, postscreen_state.c,
postscreen.h, postscreen_tests.c, postscreen.c, postscreen_smtpd.c,
postscreen_early.c.

This seems to break the postscreen cache cleanup for me. At the
scheduled cleanup time:
Jul 29 11:54:04 mgate3 postfix/master[81710]: warning: process
/usr/libexec/postfix/postscreen pid 14364 killed by signal 11
Jul 29 11:54:04 mgate3 postfix/master[81710]: warning:
/usr/libexec/postfix/postscreen: bad command startup -- throttling

and thereafter postscreen won't run unless I nuke the database, but
then signal 11 at the next scheduled cleanup.

Replacing with the unpatched snapshot restores normal operation with
the existing database.

mail_version = 3.2-20160612
postscreen_cache_cleanup_interval = 12h
postscreen_cache_map = btree:$data_directory/postscreen_cache

-- Noel Jones

Wietse Venema

2016-07-29 19:16:33 UTC

Permalink

Post by Noel Jones
This seems to break the postscreen cache cleanup for me. At the
Jul 29 11:54:04 mgate3 postfix/master[81710]: warning: process
/usr/libexec/postfix/postscreen pid 14364 killed by signal 11
/usr/libexec/postfix/postscreen: bad command startup -- throttling

Thanks for testing. The patch below should fix this.

Wietse

*** ./src/postscreen/postscreen.c- 2016-07-28 18:19:45.000000000 -0400
--- ./src/postscreen/postscreen.c 2016-07-29 15:13:48.000000000 -0400
***************
*** 838,843 ****
--- 838,844 ----
void *unused_context)
{
PSC_STATE dummy;
+ PSC_CLIENT_INFO dummy_client_info;

/*
* This function is called by the cache cleanup pseudo thread.
***************
*** 847,852 ****
--- 848,854 ----
* silly logging we remove the cache entry only after all tests have
* expired longer ago than the cache retention time.
*/
+ dummy.client_info = &dummy_client_info;
psc_parse_tests(&dummy, stamp_str, event_time() - var_psc_cache_ret);
return ((dummy.flags & PSC_STATE_MASK_ANY_TODO) == 0);
}

Noel Jones

2016-07-29 19:33:20 UTC

Permalink

Post by Wietse Venema

Thanks for testing. The patch below should fix this.
Wietse
*** ./src/postscreen/postscreen.c- 2016-07-28 18:19:45.000000000 -0400
--- ./src/postscreen/postscreen.c 2016-07-29 15:13:48.000000000 -0400
***************
*** 838,843 ****
--- 838,844 ----
void *unused_context)
{
PSC_STATE dummy;
+ PSC_CLIENT_INFO dummy_client_info;
/*
* This function is called by the cache cleanup pseudo thread.
***************
*** 847,852 ****
--- 848,854 ----
* silly logging we remove the cache entry only after all tests have
* expired longer ago than the cache retention time.
*/
+ dummy.client_info = &dummy_client_info;
psc_parse_tests(&dummy, stamp_str, event_time() - var_psc_cache_ret);
return ((dummy.flags & PSC_STATE_MASK_ANY_TODO) == 0);
}

Thanks. Seems to work as expected.

-- Noel Jones

Wietse Venema

2016-07-29 22:09:46 UTC

Permalink

Post by Wietse Venema

$ cd $POSTFIX-SOURCE
$ patch -p1 <20160728-postscreen-3.1-3.2.patch

Updated to: 29169729-postscreen-3.1-3.2.patch, which fixes some
missing code that Noel reported earlier today.

Wietse

Wietse Venema

2016-07-31 17:44:45 UTC

Permalink

I have uploaded postfix-3.2-20160730, as well as postscreen patches
for Postfix 2.11 and later (in subdirectory experimental/feature-patches).

Wietse