The mortgage company would require a gift letter for any "borrowed" down payment.
If you signed the gift letter, you need to include the money on IRS form 709, to avoid gift taxes.
This money can come out of your estate tax exemption amount of 11.58 million.
Google unified credit.