Can a Canadian perminant resident apply US perminant resident

I just had a short landing in Canada yesterday. But when I came back from Canada, US officer asked me if I applied US perminant resident. I answered "Yes". He told me it is illegal. He would put some notes in my file. Does somebody know if a canadian perminant resident can apply US perminant resident? I am really scary now. Anybody if you know, please reply it. Thanks.