UPNP fails in all stable 1.7.* releases tried to build 1.7.2 3458

[TK1]

Without error logging ticked I get no report from 1.7.* regarding UPNP mappng, with 1.6.1 it would always report success or failure regardless of logging settings... With Error logging enabled (verbose or otherwise)

I get 3 reports of

UPnP: Could not map UPnP Port on this pass, retrying.

with the attmepts terminating in

UPnP: Unable to map port xxx.xxx.xxx.xxx:yyyyy with UPnP.

Using sysinternals TCPView I am not seeing port UDP 1900 being opened for listening by utorrent when the OS UPnP is disabled as in previous versions so it is not evening listening to the UPnP device advertising broadcasts! It is not using the OS UPnP API either.

What it does do is whilst it is making those 3 tries it has open a UDP port in the 50xxx region and sends a broadcast to the UDP port 1900 used by other control points (not device hosts)

M-SEARCH * HTTP/1.1

HOST: 239.255.255.250:1900

ST:upnp:rootdevice

MAN:"ssdp:discover"

MX:3

as usual there are broadcast adverts from the 3com office connect router advertising itself as the UPnP gateway to the control point UDP port 1900 but nothing is listening in uTorrent at that port so Utorrent learns nothing about the UPnP network topology.

I'm not sure what the intention was wth this complete change in methodology maybe LAN UPnP control point peer discovery? Or maybe trying to use an infrequently implimented part of UPnP device host prototcol to avoid having to rely on binding UDP 1900 to a local interface and thus avoid conflicting with the operating systems official UPnP control point interface.

I use UPnP as I move this Vista based Laptop around various LAN's and therefore manual port forwarding is impractical.

TK1

[DreadWingKnight]

And with wireshark, are you seeing uTorrent attempt to broadcast out to identify UPnP devices?

[TK1]

As I wrote:

"What it does do is whilst it is making those 3 tries it has open a UDP port in the 50xxx region and sends a broadcast to the UDP port 1900 used normally by other control points (not device hosts)

M-SEARCH * HTTP/1.1

HOST: 239.255.255.250:1900

ST:upnp:rootdevice

MAN:"ssdp:discover"

MX:3

"

That is getting out on the wifi but not all Internet Gateway devices impliment that optional part of UPnP device host protocol. So relying solely upon that is not enough. BTW I am not using wireshark just another 3rd party packet sniffer.

[DreadWingKnight]

Well, my Linksys WRT54GL with Thibor firmware responds to the M-Search packet.

Many other vendors do implement that part of the protocol.

Cite your source for where it says that the specific part of the protocol is optional.

[DreadWingKnight]

Also, try the UPnP tester from http://noeld.com/programs.asp?cat=dstools#upnptest to see if your router gets detected by it.

[TK1]

Yup Noëls upnptest does just fine as that uses the windows SSDP Service API rather than it's own variant. His tester also detects both UPNP enabled routers I have access to here using All the MS UPNP control point implimentations from the windows 98 addon for XP ICS control to Vista native UPNP implimentation.

[DreadWingKnight]

And what SSDP related packets are your packetsniffer finding when you use noel's tester?

[TK1]

hundreds... first the the router advertises its there... then SSDP service interogates the router and registers it as one of the system networks UPNP devices, that happens before upnptest runs, when upnptest starts up and searches it uses the systems SSDP comunication layer and in response to a few router broadcasts advertisng what it is and what its http server runs on then upnptest interogates those services and obtains variables and the heirachical structure of the gateway device.

It is certainly far too much information to post here. Certainly far more data traffic than utorrent 1.6.1 when that was working as designed, since that only looked for specifc variables and commands related to NAT/NAPT port mapping and paid no attention to all the other features offered by the gateway device like link status or connection up time or bandwidth or the myriad of other features.

As I see it, the problem is utorrent 1.7.* does not make any attempt to connect to anything listening on UDP port 1900 on the local machine and does not attempt to open a listening port of it's own if no SSDP Service is running, so does not ever register the periodic anouncment broadcasts from LAN UPnP devices.

There is small comand line open source tool that seems to correctly integrate with all availabe MS UPNP implemmentations that could be used as blueprint to handle MS UPNP and the old 1.5 to 1.6.1 implementation useed when no OS support fo UPNP isdetected ie when running under win2k or wine or win95 or windows 98/ME/XP/2003/Vista with UPNP simply switched off.

It's at:

http://sourceforge.net/projects/upnpportforward/

[DreadWingKnight]

You seem to be insisting that the UPnP announcement from the router should be what uTorrent listens for in order to make its forward. My question to you on this, is WHY?

Check the SSDP packets when the upnptest application interrogates the router, and see what packets are being sent and received.

When I did the packet check with wireshark, the SSDP packets used an M-SEARCH packet, and the local port on my computer was NOT 1900. This is with the upnp tester application.

[TK1]

Before upnptest can access the UPNP dateway device to control it, it has to find out it's listening port and control xml interfaces to send the SOAP commands to, upnptest gets that via the windows SSDP service API which is listening on UDP port 1900 receiving the Announce broadcasts by attached UPNP devices including gateway devices.

If you look at the m-search broadcasts instigated by utorrent 1.7.* and upnptest you will see there is no mention of a specific port for the UPNP device to report back on becuase they are supposed to just report back with a generic UDP NOTIFY broadcast to IP 239.255.255.250 port 1900

Since windows SSDP service is sitting on that port and upnptest has registered its intrest in activity on that service by registering a callback it gets the information it is after. But since utorrent 1.7.* are NOT listening on port 1900 and have not registered their interest in that SSDP services activity it gets nothing back.

The Gateway Device NOTIFY's contain base URLs the control points can use to administer the gateway device.

e.g.

NOTIFY * HTTP/1.1

Host:239.255.255.250:1900

NT:urn:schemas-upnp-org:service:WANIPConnection:1

NTS:ssdp:alive

Location:http://192.168.1.1:80/igd.xml

USN:uuid:00000000-0000-0001-0002-00147c482ad2::urn:schemas-upnp-org:service:WANIPConnection:1

SERVER: 3Com-ADSL-11g/1.0 UPnP/1.0

Cache-Control:max-age=60

in that case dropping http://192.168.1.1:80/igd.xml into a suitable soap capable browser and you will get an XML document describing the igd further.

Yours will be similar but unique to your type of NAT router and subnet.

In most cases with outbound connections, they do not naturally occour on the listening server port. What seems to be being missed here is that upnptest does *not* function without the UPNP framework being active and accessable by the network while utorrent ever since around version 1.5 has endevoured to impliment a totally self-contained UPNP InternetGatewayDevice Control point interface.

The end result is utorrent 1.5 to 1.6.1 UPNP worked on anything from windows 95 to windows vista including windows 2000, as long as the microsoft upnp framework was *disabled* when utorrent went to look for the NAT router.

Conversely upnptest ONLY works where the mcrosoft upnp framework is up and running and in good health and in the current beta it even asks to enable any disabled services or unblock windows firewall if it is standing in the UPNP frameworks way!

UDP broadcasts like the M-SEARCH from potential control points and NOTIFY from device hosts are connectionless, a listener must be in place to recieve them with sufficient buffer space allocated and with utorrent 1.7.* there is no listener to the NOTIFY broadcasts from the device hosting the InternetGatewayDevice therefore it fails.

[DreadWingKnight]

I did look at the packets that came from both uT and the upnptest.

They DO specify a return port, because UDP replies normally go to the origin port.

http://depthstrike.com/~dwknight/upnp.pcap

Load it up in wireshark

Look at the SSDP packets.

You're saying uT is doing things wrong, but it isn't.

[Greg Hazel]

http://quimby.gnus.org/internet-drafts/draft-cai-ssdp-v1-03.txt

"Discovery occurs when a SSDP client multicasts a HTTP UDP discovery request to the SSDP multicast channel/Port. SSDP services listen to the SSDP multicast channel/Port in order to hear such discovery requests. If a SSDP service hears a HTTP UDP discovery request that matches the service it offers then it will respond using a unicast HTTP UDP response."

"unicast" meaning a response directly to the IP/port that sent the request. The problem with listening on 1900 is that many other applications do the same thing - so in more cases than broken router firmware, software prevents uTorrent from binding to port 1900.

[TK1]

alus yes I agree that is the problem of needing to own port 1900, this is the bugbear of ignoring OS support and _entirely_ going ones own way and implimenting the entire upnp control point like open source products tend to do for the sake of portability to linux and other non-MS operating systems. The routers that have the problems with the 1.7.* scheme are ones that simply periodicaly notify broadcast their control interfaces and work with the installed SSDP service and avoid having to run yet another server on itself to listen to other devices multicast searches or notifications.

Is there any reason why utorrent being a primarily MS windows development avoids using the built in SSDP and control point API where it is available and instead it initiates its own custom control point, including previosly owning UDP port 1900 when it requires control of the InternetGatewayDevice and all the compromises that entails.

In reality it only needs to own 1900 when no discovery framework is in place AND it needs to check the current port forwarding and restore it to the desired state. Rest of the time it could give it up so others that wish to do it themselves can have a peek at what's being multicast over the LAN and have a chance at controling something.

As an aside that draft you cite is over 7 years past due deletion date like and superceded by http://quimby.gnus.org/internet-drafts/draft-cai-ssdp-v1-04.txt

which was delted...

This "broken router firmware" operates just fine with all official control point API's including vista, but not with the current experimental broken partial control point implimentations based upon a subsection of an internet draft document intended to expire in April 2000 yet was left around for historical reference.

I guess if utorrent is to be stuck with this new upnp bodge then one just has to be thankfull for UPNPPortForward sourceforge project that can be used to bypass these experimental compromises and use the windows API like Noëls upnptest does.

[Ultima]

Is there any reason why utorrent being a primarily MS windows development avoids using the built in SSDP and control point API ...

µTorrent is targeted at all versions of Windows from 95/NT and up, not just XP and up. Can you really say that just because it's targeted for Windows, it'll automatically be guaranteed that the system will have UPnP built in? Those older OSes without native UPnP support won't work unless UPnP is implemented from within µTorrent. A fallback to the UPnP implemented in Windows XP and up was previously planned, but I'm not sure what the feature's current status is.

[DreadWingKnight]

Despite what it first appears, uTorrent is not going to be windows-exclusive for long.

Additionally, uTorrent is used on linux under WINE.

WINE has not yet implemented its own UPnP framework for applications to use. Users of uTorrent on linux in this environment NEED uTorrent's current UPnP support in order to be able to use UPnP on their system.

You still haven't cited your source that says that M-SEARCH packet responses are optional.

[TK1]

DreadWingKnight the wireshark log URL you invite me to look at returns a Forbidden error from the webserver.

With upnptest and upnpportforward and others, the way the control point informs the IGD where it's web server is using the SUBSCRIBE packet registering a callback containing the return URL including port number.

Regarding my asserton that the entire M-Search part is in itself optional, a document that demonstrates it also states in its preamble:

Internet-Drafts are draft documents valid for a maximum of six

months and may be updated, replaced, or obsoleted by other documents

at any time. It is inappropriate to use Internet- Drafts as

reference material or to cite them other than as "work in progress."

The draft document describes 3 models, the pure M-SEARCH based, the Pure NOTIFY based and a combined model that announces its presssence when it comes online a few times then shuts up unless it is M-SEARCH searched for or if it is being taken offline then it send a byebye NOTIFY multicast.

Current utorrent will not work on the pure NOTIFY ones like mine. Prior to 1.7.* stables it did work just fine as long as I temp disabled windows UPNP using GRC.COM UnPlugnPray utility.

If you don't understand the pure NOTIFY model, it's simple the IGD periodically multicasts a selection of NOTIFY packets describing it's control interface (http://IP:port/interface.xml) any control point interested can then query any of those xml documents and once it knows it wants to control the IGD in any way it then sends a SUBSCRIBE packet to IGD directly registering it's own web server interface as the callback... from then onwards it talks direct to the IGD custom web server to control it or poll it for IGD status.

No M-SEARCH is required or responded to in the pure NOTIFY model, but official control points will throw one out there to wake up Netgears etc that go to sleep after the initial switch on NOTIFY period expires.

That is based upon live research in working IGD's not long expired draft documents that should not be cited as anything other than work in progress according to its authors.

FYI UPNP control point API is aftermarket installable on windows98 as part of XP Internet Connection Sharing , it comes in a small setup executble distributed with XP SP2. Windows ME has always had it as part of it's distribution. Windows NT prior to XP lacked it though. I suspect it is unlikely that th windows98 package could be made to run under windows95 but as I don't have a live 95 system here I cannot check it.

I think it is odd that the official OS support for UPNP for at least 4 major versions of windows should be considered the fallback option. I would think the OS support should be the mainstream UPnP interface and when that is missing or disabled a cut down control point should be the fallback!

It might be possible to install the windows 98/ME UPnP control point framework into wine, the only problem i forsee is last time I used wine it did not have a task bar with a systray section so there would be no place for the UPnP gateway devices notification/control icon to reside. That would mean manual control and status viewing would be impossible but the API itself might still function even if the GUI did not!

[DreadWingKnight]

DreadWingKnight the wireshark log URL you invite me to look at returns a Forbidden error from the webserver.

Fixed, forgot to chmod 644 it.

You have not cited your source on the M-SEARCH packets sufficiently to help us identify who is right or wrong.

Link it, cite the section of the document, and quote it.

Additionally, since you are the only one complaining about this issue, and are ONLY having this issue on 3com routers, where we are not having this issue on DLink, Netgear, Belkin or Linksys (now Cisco Home and Small Business) routers, we can only run on the assumption that the 3com routers are running with broken firmware due to the "de-facto standard" situation we are currently in.

And where were you when this thread was active?

[TK1]

It's not a right or wrong issue but two incomplete implimentations, utorrent 1.7.* is incomplete in that it has ceased using the NOTIFY multicasts as it did previously and the 3com and some billion routers that use the regular periodic multicast notify method only rather than the M-search or hybrid method.

I don't believe I am the only having the issue and I am sure there are other posts commenting on UPnP no longer working since 1.7.* for other makes of ADSL routers eg I know Billion BiPac based firmware also uses the same scheme and would fail with 1.7.* but work fine with 1.5 to 1.6.1 if I put that back in service again.

Where was I when that thread, id 10373 was active? elsewhere obviously!

Ok if you want an old document on UPnP which has the information within it but is not a defacto standard like say the patent application itself...

http://quimby.gnus.org/internet-drafts/draft-cai-ssdp-v1-03.txt

--begin quote--

2.3.1.3. Why does SSDP support both service discovery requests as well

as service presence announcements?

Some discovery protocols only support discovery requests, that is,

the client must send out a request in order to find out who is

around. The downside to such solutions is that they tend to be very

expensive on the wire. For example, we want to display to our user

all the VCRs in her house. So we send out a discovery request.

However our user has just purchased a new VCR and, after starting

our program, plugged it in. The only way we would find out about the

new VCR and be able to display it on our user's screen is by

constantly sending out discovery requests. Now imagine every client

in the network having to send out a torrent of discovery requests

for service they care about in order to make sure they don't miss a

new service coming on-line.

Other systems use the opposite extreme, they only support

announcements. Therefore, when our user opens the VCR display window

we would just sit and listen for announcements. In such systems all

the services have to send out a constant stream of announcements in

order to make sure that no one misses them. Users aren't the most

patient people in the world so each service will probably need to

announce itself at least every few seconds. This constant stream of

traffic does horrible things to network efficient, especially for

shared connections like Ethernets.

SSDP decided to adopt a hybrid approach and do both discovery and

announcements. This can be incredibly efficient. When a service

first comes on-line it will send out an announcement so that

everyone knows it is there. At that point it shouldn't ever need to

send out another announcement unless it is going off-line, has

changed state or its cache entry is about to expire. Any clients who

come on-line after the service came on-line will discover the

desired service by sending out a discovery request. The client

should never need to repeat the discovery request because any

services that subsequently come on-line will announce themselves.

The end result is that no one needs to send out steady streams of

messages. The entire system is event driven, only when things change

will messages need to be sent out. The cost, however, is that the

protocol is more complex. We felt this was a price worth paying as

it meant that SSDP could be used successfully in fairly large

networks.

2.3.1.4. Doesn't the caching information turn SSDP back into a

"announcement driven" protocol?

Discovery protocols that only support announcements generally have

to require services to send announcements every few seconds.

Otherwise users screens will take too long to update with

information about which services are available.

SSDP, on the other hand, allows the service to inform clients how

long they should assume the service is around. Thus a service can

set a service interval to seconds, minutes, days, weeks, months or

even years.

Clients do not have to wait around for cache update messages because

they can perform discovery.

--End quote--

Note the document does not say one device type is wrong and the other is right it just states that is how the different devices are.

Windows SSDP uses the hybrid approach described above and works with NOTIFY announce only and M-SEARCH only as well as variations of those two otherwise exlcusive protocols combined.

Currently as of utorrent 1.7.* it only performs the M-SEARCH discovery requests and no longer supports the announce protocol as it did in 1.5 to 1.6.1. implimenting only part of the SSDP is in itself not wrong but is also not right either it is just a compromised design choice.

The proper intended method is to use the built in SSDP service that the windows UPnP control point API offers and upnptest uses to the fullest.

When that service is not available then a cut down (as in only looks for and understands port forwarding on IGD's) hybrid SSDP that supports both M-SEARCH polling and NOTIFY anouncing is the optimal choice in my opinion.

I'd wager that in the vast majority of cases where utorrent 1.5 to 1.6.1 found UDP port 1600 already in use is when they were running in a version of windows that had the UPnP framework enabled and up and running and therefore fully avaialble for use by any application willing to follow the published API.

[Falcon4]

I say we put TK1 in a development position.

It's about goddamn time someone brought up the 110% nonfunctional post-1.5 UPnP implementation up in technical detail. I'm so fucking tired of UPnP doing nothing more than looking pretty in any of hundreds of router models I've tested it over... I don't even rely on it anymore. It just don't work. Period.

[TheShadowRunner]

uPnP is fucked here too since version newer than 1.6, it's getting annoying to restart utorrent each time to "refresh" it.

[TK1]

If you have Win98/SE with the windows XP UPNP upgrade or windows ME or XP SP2 or Vista with UPNP framework or device discovery enabled then you can get the final stable alpha release of upnpportforward from http://sourceforge.net/projects/upnpportforward/ and drop the exe into the same folder as your utorrent and create a shortcut to upnpportforward.exe using the following command line, where PPPP is the decimal port you need forwarding:

UPNPPortForward.exe utorrent.exe tPPPP

And disable UPnP within utorrent until such time it can be fixed.

When you exit from the above utorrent session, the port mapping will be removed.

If you have windows 95 or NT3 to 4 or windows 2000 or wine the only solution I can currently offer is swap back to an earlier version of utorent that works for you and your UPnP enabled router. Until such time utorrent embraces the entire SSDP both internally and hopefully via windows too.

[cs96and]

UPnP has stopped working for me since 1.7 too. I've just done a quick wireshark comparison of 1.6 and 1.7 and the difference seems to be that 1.6 would send a broadcast to 239.255.255.250 and a unicast directly to my router. 1.7 only sends the broadcast. My router (a Zoom X6) seems to respond to the unicast message, but not the broadcast message.

What is the reason behind this change? I don't know much about SSDP/UPnP so most of the above conversation is gibberish to me.

[Ryan Norton]

> As I see it, the problem is utorrent 1.7.* does not make any attempt to connect to anything listening on UDP port 1900 on the local machine and

> does not attempt to open a listening port of it's own if no SSDP Service is running, so does not ever register the periodic anouncment broadcasts

> from LAN UPnP devices.

The differences between 1.6 to 1.7 are:

1) It binds to a random port instead of 1900

- in most cases it was because people were using a VPN that decided to take 1900 for itself, not that the Windows UPNP was enabled

2) Removal of the unicast send to the router (I wasn't involved in this decision but the general idea was that it was outdated and not used by routers)

3) three broadcasts several times instead of one to avoid congestion problems

Next time I get a chance to work on this maybe if TK1 wants to correspond via IRC or whatnot to sort out the real issues that would be good.

[TK1]

Yes Ryan, time permitting I'd be willing to help bounce a few ideas about regarding how we might improve the UPNP operability accross the platforms, bear in mind that utorrent is a tool for me not an obsession so I do not tend to lurk in the forums (hence me missing the prevoius UPNP deleopment discussion) Some proactive notification that the time had arrived would be advisable ie an email notification or I could stay subscribed to this thread and when you are ready to start discussions in IRC or here, you can post a reply here which will get seen by me.

Any idea what badly written win32 VPN server it is which is owning UDP port 1900? If it is an open source or in current development, project then maybe they could be brought into the utorrent development phase...

Going back to the text I was forced to quote, that seems to be one of the final remaining documents describing the functionality of UPNP SSDP.

It was a draft RFC, which is inviting suggestions and comments.

I have one sugestion, add to the non-win32API UPNP framework specification that if a third party application needs to control or determine the status of an installed Internet Gateway Device, then it should test to see if UDP port 1900 is taken if it is then it should wait a pre-assigned time-out and try the port again, when it gets ownership of UDP then it should cache any Announce broadcasts, and while it is there it can broadcast it's own targeted SEARCH request from a none 1900 port and cache any unicast replies, once it has determined an IGD exists, it should be able to release UDP 1900 for other UPNP control-points to have a loan of UDP 1900.

Using the cache'd data from the IGD it should be a (relatively) simple matter to unicast to the IGD and get it's status and control interface and use it.

Once the device and its interfaces are found there should be no more need for the local UDP 1900, once the 3rd party control point has subscribed to the IGD it only has to poll it's status to make sure it is still there and it has not been rebooted since last time the port mapping was set.

Using this style of co-operative UDP port 1900 time-sharing then multiple cross platform 3rd party UPNP enabled programs could co-exist on the same system

By simple act of testing the win32api UPNP SSDP control point interface, if it succedes, use it, if not then use the above co-operative hybrid 3rd party SSDP, I suspect if that is implimented fully in utorrent, then the vast majority of users with a UPNP enabled IGD will be plesantly supprised regardless of win32 version...

Another possible approach could be to create a pseudo network interface on a different subnet like the 224.0.0.x subnet created for windows media TV interface in some configurations, if you can create it on the fly so windows or this rogue VPN server application cannot bulk bind to it, then you could open UDP 1900 only on your pseudo network interface, harvest all the broadcasts you want and then subscribe to the IGD using the normal IP unicast.

I recall under windows 98 I would have utorrent bind to the main LAN IP (192.168.x.x) and then enable UPNP in windows after utorrent had loaded. In that configuration win32API SSDP UDP 1900 bound to the 224.0.0.4 IP and 127.0.0.1 and both utorrent AND windows were able to see and control the 3com IGD! It does work under Vista too but because I move this Vista laptop around various networks in my work and play I cannot do the same trick as the LAN side IP number changes for each LAN subnet.

[Greg Hazel]

Is there any reason why utorrent being a primarily MS windows development avoids using the built in SSDP and control point API where it is available and instead it initiates its own custom control point, including previosly owning UDP port 1900 when it requires control of the InternetGatewayDevice and all the compromises that entails.

Aside from support for other platforms, it's important to note that the WinXP built-in UPnP service is often broken. With my router, if there are more than a few entries in the list of mapped ports the XP service gives up entirely, where the uTorrent system still works.

As a side note, NAT-PMP always works great for me, and is a million times more simple on the router side to implement.