[arcon]

I was finally convinced, by someone on this forum actually, to run an openBSD firewall. Being a long time Linux advocate I must say I am impressed with the os. However, I am having a problem I can't seem to solve that I assume invloves pf. Here goes:

First, for reference here's my pf.conf, blocks were commented out to recduce possible problems in solving this, and ips have been changed to protect the innocent:
ext_if="rl1"
int_if="rl0"
internal_net="10.1.1.0/24"
external_addr="111.111.111.12"
scrub reassemble tcp fragment reassemble no-df random-id
scrub in all min-ttl 5
scrub out all min-ttl 30
nat on $ext_if from $internal_net to any -> $external_addr
pass in all
pass out all
(initially I had no scrubbing options, then scrub all, but neither seemed to help. I set these scrub options as a result of reading Absolute OpenBSD's pf chapters)

I have two static ips over my dsl at home, no ppoe, where I have a web server (linux) and a firewall doing NATp for my home network each on one of those ips. I am trying to connect to a mail server in colo behind an openbsd firewall doing straight NAT to the mail server. The mail server is runing spop, simap, ssh, and https on port 1022. I can connect via ssh to this or another box behind the same firewall and login, however if I try and issue a command which spews large amounts of data onto the screen (ie. psauxw) I am disconnected before I get any output. Also, I try and connect via https and it hangs. Trying either ssh or https from the linux box works perfectly, I do not have a problem loading pages, issuing commands, etc. I ran tcpdump on both the openbsd and the linux box and the output shows the first packet openbsd is 'missing' is a 1024 byte packet. This, along with the ssh funniness is what makes me think it is a mtu/don't fragment issue. Here are copies of the tcpdump output for loading the https page locally off each box:

openBSD:
# tcpdump -tS -i rl1 host remote
tcpdump: listening on rl1
local.2115 > remote.1022: S 2999345758:2999345758(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 221453162 0>
remote.1022 > local.2115: S 4222525973:4222525973(0) ack 2999345759 win 0 <mss 1400> (DF)
local.2115 > remote.1022: . ack 4222525974 win 16800
remote.1022 > local.2115: . ack 2999345759 win 32200 (DF)
local.2115 > remote.1022: P 2999345759:2999345885(126) ack 4222525974 win 16800
remote.1022 > local.2115: . ack 2999345885 win 32200

linux:
server root # tcpdump -tS -i eth0 host remote
tcpdump: listening on eth0
local.34799 > remote.1022: S 3354073354:3354073354(0) win 5840 <mss 1460,sackOK,timestamp 97897212 0,nop,wscale 0> (DF)
remote.1022 > local.34799: S 3791498540:3791498540(0) ack 3354073355 win 0 <mss 1400> (DF)
local.34799 > remote.1022: . ack 3791498541 win 5840 (DF)
remote.1022 > local.34799: . ack 3354073355 win 32200 (DF)
local.34799 > remote.1022: P 3354073355:3354073469(114) ack 3791498541 win 5840 (DF)
remote.1022 > local.34799: . ack 3354073469 win 32200
remote.1022 > local.34799: P 3791498541:3791499565(1024) ack 3354073469 win 32200
local.34799 > remote.1022: . ack 3791499565 win 7168 (DF)
remote.1022 > local.34799: P 3791499565:3791499645(80) ack 3354073469 win 32200
local.34799 > remote.1022: . ack 3791499645 win 7168 (DF)
local.34799 > remote.1022: P 3354073469:3354073659(190) ack 3791499645 win 7168 (DF)
remote.1022 > local.34799: P 3791499645:3791499696(51) ack 3354073659 win 32200
local.34799 > remote.1022: . 3354073659:3354075059(1400) ack 3791499696 win 7168 (DF)
local.34799 > remote.1022: P 3354075059:3354075096(37) ack 3791499696 win 7168 (DF)
remote.1022 > local.34799: . ack 3354075096 win 32200
remote.1022 > local.34799: P 3791499696:3791499757(61) ack 3354075096 win 32200
remote.1022 > local.34799: P 3791499757:3791501157(1400) ack 3354075096 win 32200
local.34799 > remote.1022: . ack 3791501157 win 9800 (DF)
remote.1022 > local.34799: FP 3791501157:3791501667(510) ack 3354075096 win 32200
local.34799 > remote.1022: F 3354075096:3354075096(0) ack 3791501668 win 12600 (DF)
remote.1022 > local.34799: . ack 3354075097 win 32200

Some more background. Recently we had a problem with the openbsd box in front of the mail server preventing users behind DSL from accessing the servers. The problem turned out to be it was setting the mtu to 1460 and setting the don't fragment bit. However, the ppoe header takes up the first 60 byes of the packet, making the effective max mtu 1440. Changing this fixed the problem there. I have also tried setting my mtu in pf and on both interfaces to 576, 1440, 1024, etc. but haven't seen any of it work. I am *extremely* confused and even tried voodoo tactics like swapping the linux and openbsd boxen's ip addresses to see if that would help. What am I missing?

-Adam

[|MiNi0n|]

Try commenting out as much as you possibly can from each box. Try a basic open ruleset and see if you still have trouble:

pass in all
pass out all

And go from there. I've never seen this issue myself before, perhaps others here have but I imagine it'll be quiet for a day or two... considering it's Thanksgiving time for all you darn Yanks :lol:

What are you running, 3.4?

[arcon]

Speak of the devil. Hey man, it's Adam from Atlanta. You are one of the culprits that got me this new firewall I am struggling with. :)

Anyways, me and the guy with the other firewall are trying to get a moment together to play around with both sides at the same time. I currently have what you mention setup on my end now that I have found something in my current scrub setup blocks a few random sites I have hit around the net. I have one config with the scrubs and one without, switching between them solves problems with other sites, but not the one OBSD firewall I *really* need to talk to. What really puzzles me is that my Linux box is reacting fine to the other firewall so I'm thinking there should be something on the OBSD box I can change to at least match the Linux box, if not really solve the problem on both ends at some point in time. I'm sure someone will point out something obvious to me soon.

And yes, I am running 3.4.

[elmore]

the scrub options can be al little diffucult to work with, in particular the scrub out option. Might check the pf faq on OpenBSD's site they have some good info. Also if you guys are on dsl mtu settings will most like need to be played around with on both the sending and receiving ends, in particular if the dsl connect is pppoe instead of atm bridged mtu settings might have to be dropped to at least 1440 both in pf and on the interface. Again the FAQ covers this and I'm more than willing to work woith you and Taylor to get this rresolved. Anyways check thr OBSD FAQ for now and when I get back into town, we'll get it straight! :)