IIS AppPool and Virtual Directory

I am running IIS 7.5 on Windows Server 2008R2.

I have an .NET4 Intranet web application with a Dedicated Pool.

I'm using ApplicationPoolIdentity and Windows Authentication.

I have a virtual directory on the web server and another one on a separate server.

I have successfully given permissions on the web server virtual directory to    IIS AppPool\<AppPool Name>  .

However try as I may I cannot give permissions to   IIS AppPool\<AppPool Name>  on the virtual directory which resides on a different server.

Many thanks