Problem trying to make my DLL callback a VBA function with paramet

[Thivierge.M]

Hi everyone,

I have been struggling for over a week now, trying to make my DLL call a
function written in VBA (version 6) when parameters are involved.

I am able to call my DLL functions from VBA without any problems. The DLL
can even call a VBA procedure (sub or function) without parameters
successfully (using the callback technique) but as soon as parameters are
passed with the call, an exception is raised in the VBA host application which
makes it crash.

I have done my homework by reading the available help files on the
subject and searching on the web without success.

Here the project I have built to demonstrate my problem:

My TestDLL.dll has been developed with Microsoft Visual C++ .NET

(1) Here is the content of DllTest.cpp:

#include "stdafx.h"

#define EXTERNC extern "C"
#define DLLAPI __declspec(dllexport)
#define WINAPI __stdcall

typedef long (CALLBACK *PCB1) (char cVal);

EXTERNC DLLAPI DWORD WINAPI DllCBInit (PCB1 pCallback);
EXTERNC DLLAPI void WINAPI DllCBTest1(char cVal);

static PCB1 theCallback = NULL;

// Defines the entry point for the DLL application.
BOOL APIENTRY DllMain(HANDLE hModule,
DWORD ul_reason_for_call,
LPVOID lpReserved)
{
return TRUE;
}

// Exported functions of the DLL
EXTERNC DLLAPI DWORD WINAPI DllCBInit(PCB1 pCallback)
{
theCallback = pCallback;
return (DWORD)theCallback;
}

EXTERNC DLLAPI void WINAPI DllCBTest1(char cVal)
{
(*theCallback)(cVal);
}

(2) Due to C++ name mangling, I have defines the function names in the
DLLTest.def file as follows:

LIBRARY DllTest

EXPORTS
DllCBInit
DllCBTest1

(3) I compiled the project DllTest with the Calling Convention __stdcall (/Gz)
as suggested everywere in the documentation. Then I copied the DllTest.dll
file
in c:\windows\system32

(4) In a VBA script (written under Excel 2003 for convenience purposes) I have
created a Main module with the following code:

' Force explicit variable definition
Option Explicit

Public theByte As Byte

Public Declare Function DllCBInit Lib "DllTest" (ByVal pCallback As
Long) As Long

Public Declare Sub DllCBTest1 Lib "DllTest" (ByVal cVal As Byte)

Public Sub Callback1(cVal As Byte)
On Error Resume Next 'to prevent error being propagated back to the caller
theByte = cVal
End Sub

Public Sub TestDLLCallback()
Dim lStatus As Long
'Initialize the callback
lStatus = DllCBInit(AddressOf Callback1)
Debug.Print lStatus
'Test the Dll callback functionality
Call DllCBTest1(128)
Debug.Print theByte
End Sub

(5) Here when I step through the TestDLLCallback() procedure, the first debug
print displayed 59111236 (0x0385f744) as the address of the VBA callback
procedure Callback1.

(6) Then if I step through the Call DllCBTest1 the cursor goes to the
Callback1() procedure as expected. The next step causes the application to
crash.

(7) I used the Visual Studio.NET IDE to attach to the EXCEL application and
set a breakpoint in the DLL at the DllCBTest1 function entry point. I stepped
through the assembly code to finally realize that the application crashes
when trying to access the content of an invalid memory location:

651FAF5E 8B 00 mov eax,dword ptr [eax] ---> where eax = 0xCCCCCC80

The content of eax seems to be the actual value of the passed parameter (128)
stored as one byte in the register eax and pushed onto stack. For some unknown
reason, this value comes back from the stack to haunt us. The assembly code
for DllCBTest1 is as follows:

EXTERNC DLLAPI void WINAPI DllCBTest1(char cVal)
{
05741FF0 55 push ebp
05741FF1 8B EC mov ebp,esp
05741FF3 81 EC C0 00 00 00 sub esp,0C0h
05741FF9 53 push ebx
05741FFA 56 push esi
05741FFB 57 push edi
05741FFC 8D BD 40 FF FF FF lea edi,[ebp-0C0h]
05742002 B9 30 00 00 00 mov ecx,30h
05742007 B8 CC CC CC CC mov eax,0CCCCCCCCh
0574200C F3 AB rep stos dword ptr [edi]
(*theCallback)(cVal);
0574200E 8B F4 mov esi,esp
05742010 8A 45 08 mov al,byte ptr [cVal]
05742013 50 push eax ---> 0xCCCCCC80
05742014 FF 15 40 6B 76 05 call dword ptr [theCallback (5766B40h)]
0574201A 3B F4 cmp esi,esp
0574201C E8 61 F5 FF FF call @ILT+1405(__RTC_CheckEsp) (5741582h)
}
05742021 5F pop edi
05742022 5E pop esi
05742023 5B pop ebx
05742024 81 C4 C0 00 00 00 add esp,0C0h
0574202A 3B EC cmp ebp,esp
0574202C E8 51 F5 FF FF call @ILT+1405(__RTC_CheckEsp) (5741582h)
05742031 8B E5 mov esp,ebp
05742033 5D pop ebp
05742034 C2 04 00 ret 4

We can see that the parameter cVal is saved as a byte in al (byte portion of
eax).

From what I have read on the subject, there seems to be a calling convention
clash. Since I only have control in my DllTest library, I even tried to
recompile it with either __cdecl and __fastcall. I obtain same results.

The only explanation must be something silly I just can seem to figure out.

I hope that someone would be able to see what my problem is and provide
me with the solution.

I have to appologize for the lenghty description but I assumed that
the more you have to work with the easier it will be to find a solution.

I am running under WinXP Pro 2002 with service pack 1.

Thanks in advance,
Michel

 

[Thivierge.M]

Its me again. I do not know if anybody had a chance to look at my problem,
but I finally found the source of my problem. In fact there are two dinstinct
problem.

First, the reason why I couldn't pass parameters through a callback between
my DLL and VBA is only because I forgot to add the "ByVal" keyword in front
of the variables in the VBA callback function. This solved the main problem.

The other problem I faced with my real application, crashing as soon as a
callback is made, is related to the fact that the callback originates from a
parallel thread (not the main VBA thread). It seems that a parallel thread
cannot call properly a function in VBA which runs under the hosts main thread.

If someone has an idea on how to solve this problem, I will appreciate any
help.

Right now, the way I am planning to solve the problem is by calling from VBA
a function in my DLL which will perform any pending callbacks... This should
work OK! However, there could be delays between the time a callback is posted
and the time it is processed.

Regards,
Michel