Paul, thank you for your answer and for clarifying things. 
I think that code used in the question is incorrect.
I tried to develop example similar to the one in the question, so I created two stateless beans:
Code: Select all
@Stateless
public class MyStateless {
    @PersistenceContext
    private EntityManager em;
    @EJB
    private MyService myService;
    public void createAndRemoveAddress() {
        Address address = new Address();
        address.setCity("Warszawa");
        em.persist(address);
        em.flush();
        myService.removeAddress(address);
    }
}
 
Code: Select all
@Stateless
public class MyService {
    @PersistenceUnit
    private EntityManagerFactory emf;
    @TransactionAttribute(value = TransactionAttributeType.MANDATORY)
    public void removeAddress(Address address) {
        EntityManager entityManager = emf.createEntityManager();
        entityManager.remove(address);
    }
}
As you see the semantics of MyService.removeAddress are pretty the same as in the question.
When I called MyStateless.createAndRemoveAddress() in client code I got IllegalStateEception:
Caused by: java.lang.IllegalArgumentException: Removing a detached instance model.Address#0
despite that there is no call to detach() and transaction still proceed.
My conclusion (from this experiment and from our discussion) is that application-managed entity manager has completely different persistent context from the transaction-scoped persistent context created before in JTA transaction, so even if there is an entity managed by the container-managed persistent context, it is seen as detached from the perspective of application-managed entity manager. So I think you should reformulate the example to not to use the app-managed persistent context. 
There is no case when removeStudent() will correctly remove managed Student entity.