OSB and oracle BPEL both are very popular tools for integrating the systems.
After going through some blogs, here is the quick info:
1. OSB is stateless and BPEL can be made stateless by setting the audit settings. However the audit settings turning off doesn't apply to mediator.
2. OSB is far more superior in performance because it is stateless. In addition, it offers capabilities not supported in composite development such caching, throttling, and service virtualisation.
3. OSB has some severe limitations; the most serious being within transaction visibility, tracing, and metrics. It is either difficult, cumbersome, or impossible to find out if transactions have gone through the OSB engine and detailed, dynamic metrics are unavailable.
4. OSB 11g cannot reference artifacts in the MDS (MetaData Store). Custom code is required for DVM support.
5. SOA Suite provides a better capability to handle transactions, compensation and asynchronous flows.
6. BPEL is better when the logic is complex as it has many inbuilt activities.
7. You dont get any monitoring and SLA management capability with Mediator and SOA.
8. SOA Suite provides a better capability to handle transactions, compensation and Asynchronous.
9.As BPEL uses dehydration store to store the life cycle of a BPEL instance, it is very useful for long running processes.
Coming Back to OSB...
Few Features of OSB such as Service Result Caching, Message Throttling and service Virtualisation, makes it an ideal choice where Service Virtualization and
Message-Oriented Solutions are required.