We consider the problem of efficiently inferring the parameters in gene regulatory networks. Whilst it is possible to work with a discrete stochastic model for inference, computational cost can be prohibitive for networks of realistic size and complexity. By treating the numbers of molecules of biochemical species as continuous, a diffusion approximation can be used, and whilst this approach has been shown to work well for some networks, ignoring discreteness of low copy number species is unsatisfactory. Here, we consider a hybrid inference method, treating low copy number species as discrete and the remaining species numbers as continuous. The methodology uses a hybrid simulation scheme inside a recently proposed particle marginal Metropolis-Hastings (PMMH) scheme. We apply the scheme to a simple application and compare the output with a scheme for performing inference for the underlying discrete stochastic model.